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Abstract 

For a Gaussian time series with long-memory behavior, we use the 
FEXP-model for semi-parametric estimation of the long-memory param- 
eter d. The true spectral density /o is assumed to have long-memory 
parameter do and a FEXP-expansion of Sobolev-regularity /3 > 1. We 

prove that when k follows a Poisson or geometric prior, or a sieve prior 
1 

increasing at rate n i+^f* , d converges to do at a suboptimal rate. When 

1 

the sieve prior increases at rate ji^F however, the minimax rate is almost 
obtained. Our results can be seen as a Bayesian equivalent of the result 
which Moulines and Soulier obtained for some frequentist estimators. 



1 Introduction 

Let Xt, i G Z, be a stationary Gaussian time series with zero mean and spectral 
density fo{x), x e [— 7r,7r], which takes the form 



(1.1) 



where do G {^\^ \) is called the long-memory parameter, and AI is a slowly- 
varying bounded function that describes the short-memory behavior of the se- 
ries. If do is positive, this makes the autocorrelation function p{h) decay poly- 
nomially, at rate h~^^~'^'^°K^ and the time series is said to have long-memory. 
When do =0, Xt has short memory, and the case do < is referred to as in- 
termediate memory. Long memory time series models are used in a wide range 
of applicatio ns, such as hydrol ogical or financial time series; see for example 
BeranI (jl994l ) or lRobinsonI (jl994l ). In parametric approaches, a finite dimensional 
model is used for the short memory part Mo\ the most well known example is 
the ARFIM A(p,d,q) model. Th e asymptotic properties of maximum likelihood 
esti mators (iDahlhaus (1 19891) or Lieberman et al.l ( 2003 ) ) and Bayesian estima- 
tors ( Philippe and Rousseau ( 20021 )) have been established in such models and 
these estimators are consistent and asymptotically normal with a convergence 
rate of order ^Jn. However when the model for the short memory part is mis- 
specified, the estimator for d can be inconsistent, calling for semi-parametric 
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methods for the estimation of d. A key feature of semi-parametric estimators 
of the long-memory parameter is that they converge at a rate which depends 
on the smoothness of the short-memory part, and apart from the case where 
Mo is infinitely smooth, the convergence rate is smaller than ^/n. The estima- 
tion of the long-memory par ameter d can thus be c onsid ered as a non-regular 
semi-parametric problem. In Moulines and Soulier ( 20031 ) (p. 274) it is shown 

2/3-1 

that when /<, satisfies (|1.4p . the minimax rate for d is n '^f . There are fre- 
quentist estimators for d based on the period ogram that achieve this rate (see 
Hurvich et a l. (20o3) and iMouhnes and SoulieR (200^)). 



Although Ba yesian methods i n long-memor y r nodels have been widely us ed 
(see for instance IKo et al.l (|2009l) . I Jensenl (|2004l) or lHolan and McEhovl (|2010|) ). 
the literat ure on convergence pro perties of non- and semi-parametric estimators 
is sparse. Rousseau et al. (2010) (RCL hereafter) obtain consistency and rates 
for the L2-norm of the log-spectral densities (Theorems 3.1 and 3.2), but for 
d they only show consistency (Corollary 1). No results exist on the posterior 
concentration rate on d, and thus on the convergence rates of Bayesian semi- 
parametric estimators of d. In this paper we aim to fill this gap for a specific 
family of semi-parametric priors. 

We study Ba yesian estimation of d within the FEXP-model (jBeranl (|l993h . 
RobinsonI (|l995l )). that contains densities of the form 



fd.k.eix) 



-2d 



exp 



J=0 



cos{jx) 



(1.2) 



where d E (— i, i). k is a nonnegative integer and 9 e M'''+^. The factor 
exp{^*^^Q 9j cos(jx)} models the fu nction Mr, in (| 1 . 1 [) . In contrast to the origi- 



nal finite-dimensional FEXP-model (Beran ( 19931 )). where k was supposed to be 
known, or at least bounded, may have an infinite FEXP-expansion, and we 
allow k to increase with the number of observations to obtain approximations 
/ that are increasingly close to fo- Note that t he case where the true spe ctral 
density satisfies fo = fda^kafia^ is considered in Holan and McElrov ( 2010l ). In 
this paper we will pursue a fully Bayesian semi-parametric estimation of d, the 
short memory parameter being considered as an infinite-dimensional nuisance 
parameter. We obtain results on the convergence rate and asymptotic distri- 
bution of the posterior distribution for d, which we summarize below in section 
11.21 These are to our knowledge the first of this kind in the Bayesian literature 
on semi-parametric time series. First we state the most important assumptions. 



1.1 Asymptotic framework 

For observations X = (Xi, . . . , Xn) from a Gaussian stationary time series with 
spectral density /, let Tn{f ) denote the associated covariance matrix and /n(/) 
denote the log-likelihood 

n \ 1 

Uf) = -2 log(27r) - -logdet(r„(/)) - -X'T-\f)X. 
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We consider semi-parametric priors on / based on the FEXP-model defined 
by (jl.2[) . inducing a parametrization of / in terms of {d, k, 9). Assuming priors 
TTd for d, and, independent of d, iVk for k and TTg\i^ for 6\k,we study the (marginal) 
posterior for d, given by 

n(i C I?\X) "^^^^ e^-('^'^'<^)dne^,{0)dMd) ^^^^ 

EZo ^kik) f\ 4,^, eiMe)d^e\k{e)d^M ' 

The posterior mean or median can be taken as point-estimates for d, but we 
will focuss on the posterior n((i|X) itself. 

It is assumed that the true spectral density is of the form 

{oo 
j=o 

oo 

60 e e{l3,Lo) = {ee him : + < Lo}, 

j=o 

for some known /3 > 1. 

In particular, we derive bounds on the rate at which n(c? S D\X) concen- 
trates at do, together with a Bernstein - von -Mises (BVM) property of this 
distribution. The posterior concentration rate for d is defined as the fastest 
sequence a„ converging to zero such that 

Il{\d - do\ < Kan\X) ^ 0, for a given fixed K. (1.5) 



1.2 Summary of the results 

Under the above assumptions we obtain several results for the asymptotic dis- 
tribution of n(c? g D\X). Our first main result fTheorem l2.ip states that under 
the sieve prior fc„ ~ (n/logn)^/'^^^), Il{d S D\X) is asymptotically Gaussian, 
and we give expressions for the posterior mean and the posterior variance. A 
consequence (Corollary [531 of this result is that the convergence rate for d un- 

2/3-1 _ _ 

der this prior is at least (5„ = (n/logn) , i.e. in p.5p is bounded by i5„. 
Up to a logn term, this is the minimax rate. 

By our second main result (Theorem 12. 2p . the rate for d is suboptimal 
when k is given a a Poisson or a Geometric distribution, or a sieve prior 
fc„ ^ (n/ logn) 1+2/3 . More precisely, there exists /o such that the posterior 
concentration rate a„ is greater than and thus suboptimal. 

Consequently, despite having good frequentist properties for the estimation of 
the spectral density / itself (see RCL), these priors are much less suitable for 
the estimation of d. This is not a unique phenomenon in (Bayesian) semi- 
parametric estimation and is encountered for instance in the estimation o f a 



linear functi onal of the signal in white- noise models, see lLi and Zhaol (|2002l) or 
Arbell (|2ninl ). 
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The BVM property means that asymptotically the posterior distribution 
of d behaves like a~^{d — d) ~ Af{0,l), where d is an estimate whose fre- 
quentist distribution (associated to the parameter d) is Af{do,a'^). We prove 
such a property on the posterior distribution of d given k = k„. In regular 
parametric long-memory mod els, the BVM property has been established by 



Philippe and RoussearJ (j2002[ ). It is however much more difficult to establish 



BVM theorems in infinite dimensional se tups, even f or ind e pendent and id enti- 
cally distributed models ; see fo r instance iFreedman (ll999l) . (Castiilol (I2OIOI) and 
Rivoirard and Rousseau ( 20101 ). In particular it has been proved that the BVM 
property may not be valid, even for reasonable priors. The BVM property is 
however very useful since it induces a strong connection between frequentist and 
Bayesian methods. In particular, it implies that Bayesian credible regions are 
asymptotically also frequentist confidence regions with the same nominal level. 
In section [5] we discuss this issue in more detail. 



1.3 Overview of the paper 

In section [21 we present three families of priors based on the sieve model defined 
by (|1.2p with either k increasing at the rate (71/ logn)^/^^'^\ k increasing at the 
rate (n/ logn)^/'^^^+^^ or with random k. We study the behavior of the posterior 
distribution of d in each case and prove that the former leads to optimal frequen- 
tist procedures while the latter two lead to suboptimal procedures. In section [3] 
we give a decomposition of Il{d S D\X) defined in p.3p . and obtain bounds for 
the terms in this decomposition in sections 13.21 and 13.31 Using these results we 
prove Theorems 12.11 and 12.21 in respectively sections 2] and \5\ Conclusions are 
given in section [51 In the appendices we give the proofs of the lemmas in section 
[HI as well as some additional results on the derivatives of the log-likelihood. The 
proofs of various technical results can be found in the supplementary material. 
We conclude this introduction with an overview of the notation. 



1.4 Notation 

The m-dimensional identity matrix is denoted /„• We write \A\ for the Frobe- 
nius or Hilbert-Schmidt norm of a matrix A, i.e. \A\ = vtivL4*, where A^ 
denotes the transpose of A. The operator or spectral norm is denoted = 
sup||3,||^]^ x*A*Ax. We also use || • || for the Euclidean norm on R*"' or ^^(N). The 
inner-product is denoted | • |. We make frequent use of the relations 

\AB\ = \BA\ < \\A\\ ■ \B\, \\AB\\ < \\A\\ ■ < \A\ < V^\\A\\, 

MAB)\ = \ti{BA)\ < \A\ ■ \B\, \x'Ax\ < x'x\\A\\, ^ ' ' 



see [Pahlhausl (jl989[ ). p. 1754. For any function h G Li([-7r, tt]), r„(/i) is the 



matrix with entries /^^e*'' ™l^/i(x)(ia;, /, m = 1, . . . , n. For example, T'„(/) is 
the covariance matrix of observations X — (Xi, . . . , X„) from a time series with 
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spectral density /. If h is square integrable on [— vr, tt] we note 



/TT 
-TT 



h^{x)dx. 



The norm / between spectral densities / and g is defined as 
KLg) (log/(^) - ^oggix))'dx. 

Unless stated otherwise, all expectations and probabilities are with respect to 
Po, the law associated with the true spectral density /q. To avoid ambiguous 
notation (e.g. versus 0o,o) we write 60 instead of ^o- Related quantities such 
as fo and do are also denoted with the o-subscript. 

The symbols op and Op have their usual meaning. We use boldface when 
they are uniform over a certain parameter range. Given a probability law 
P, a family of random variables {VFrfjdeA and a positive sequence a„; Wd = 
op(a„, A) means that 



P sup |Wd|/a„ > e] ^ 0,{n ^ 00). 
\deA J 

When the parameter set is clear from the context we simply write op(a„). 
In a similar fashion, we write o(a„) when the sequence is deterministic. In 
conjunction with the op and Op notation we use the letters 5 and e as follows. 
When, for some r > and a probability P we write Z — Op{n^~'^), this means 
that Z = 0{n^+^) for all e > 0. When, on the other hand, Z = Op(n^-''), we 
mean that this is true for some 5 > 0. If the value of 5 is of importance it is 
given a name, for example 5i in Lemma 13.41 

The true spectral density of the process is denoted fo- We denote /c-dimensional 
Sobolev-balls by 

Qk{P,L) = |(?eM^-+i :^0|(l+j)2'5 <i| cM^'+i. (1.7) 

For any real number x, let a;+ denote max(0, a;). The number denotes the 
sum X]j>fc+i.?~^- tie the sequence defined by rjj = — 2/j, j > 1 and 

770 = 0. For an infinite sequence u = {uj)j':>Q, let it[fc] denote the vector of the 
first k + 1 elements. In particular, ?7[fc] = {rjQ, ...,77^). The letter C denotes any 
generic constant independent of Lg and L, which are the constants appearing 
in the assumptions on and the definition of the prior. 

2 Main results 

Before stating Theorems 12.11 and 12 . 21 in section [231 we state the assumptions on 
fo and the prior, and give examples of priors satisfying these assumptions. 
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2.1 Assumptions on the prior and the true spectral den- 
sity 

We assume observations X = (Xi,...,X„) from a stationary Gaussian time 
series with law Pq, which is a zero mean Gaussian distribution, whose covariance 
structure is defined by a spectral density satisfying (|1.4p . for known (3 > 1. 
It is assumed that for a small constant t > 0, do G [~5 + *j 5 ~ 

Assumptions on H. We consider different priors, and first state the as- 
sumptions that are common to all these priors. The prior on the space of 
spectral densities consists of independent priors tt^, -Kk and, conditional on k, 
TTgife. The prior for d has density tt^ which is strictly positive on [— ^ + 1, ^ — t], 
the interval which is assumed to contain do, and zero elsewhere. The prior for 
9 given k has a density tt^i^ with respect to Lebesgue measure. This density 
satisfies condition Hyp(/C, Cq, /3, Lq), by which we mean that for a subset /C of 
N, 

min inf e'='''='°s'=7reife(6i) > 1, 

where Lq is as in (|1.4p . The choice of JC depends on the prior for k and 9\k. We 
consider the following classes of priors. 

• Prior A: k is deterministic and increasing at rate 

kn = [kA{nl\ogn)^\, (2.1) 

for a constant k^ > 0. The prior density for 6\k satisfies Hyp({kn}, Cq, — 
i,Lo) for some cq > and has support ~ In addition, for all 

9,9' e efe(^- such that \\9 - e'\\ < L(n/ log n)"^ , 

logneikiO) - log7re|fc(0') = hii9 - 9') + o(l), (2.2) 

for constants C, po > and vectors h^. satisfying \\h],\\ < C{n/kY~P°. 
Finally, it is assumed that L is sufficiently large compared to Lg. 

• Prior B: A: is deterministic and increasing at rate 

fcl = [fcs(n/logn)TWj, 

where fc^ is such that k^ < k^ for all n. The prior for 9\k has density 77^^. 
with respect to Lebesgue measure which satisfies condition IIyp({kjj}, cq, /3, Lo) 
for some co > and is assumed to have support Qk{P,L). The density 
also satisfies 

^og'^e\k{0) - log7ren.(6'') = oil), 

for all 61,6'' e efc(/3,L) such that \\9-9'\\ < L{n/ lognyw+r . This condi- 
tion is similar to (|2.2p . but with h^. = 0, and support Qk{P, L). 

• Prior C: fc - -Kk on N with e-'=i'='°sfc < ^^^(fc) < e-'=^'''°s''^ for k large 
enough, where < ci < C2 < +00. There exists /S^ > 1 such that for all 
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/? ^ Ps, the prior for d\k has density irg^k with respect to Lebesgue measure 
which satisfies condition Hyp({k < ko(n/ logn)^/'-^^"''^-'}, Cq, /3, Lq), for all 
fco > and some cq > 0, as soon as n is large enough. It has support 
included in Qkil3,L) and satisfies 

logng^kiO) - logng\k{e') = 0(1), 

for all e,e' e ekW,L) such that \\e - e'\\ < L{n/\ognyw^ . 
Note that prior A is obtained when we take /?' = /3 — ^ in prior B. 



2.2 Examples of priors 

The Lipschitz conditions on logTr^ij, considered for the three types of priors are 
satisfied for instance for the uniform prior on 9fc(/3 — \tL) (resp. Gfe(/3,L)), 
and for the truncated Gaussian prior, where, for some constants A and a > 0, 



(^)exp -A^r^ 
V J=0 

In the case of Prior A, the conditions on logTre^, and hk in (|2.2p are satisfied 
for a < 4/3 - 2. To see this, note that for all 9, O' e Qh{l3 - 1/2, L), 

j=o 

In the case of Prior B and and Prior C we may choose a < 2/3, since for some 
positive ko 

for all k < fco(n/logn)i/(2^+i) and all 0,9' G efe(/3,L) such that \\e - 6' \\ < 
(n/logn)-'3/(2/3+i). 

Also a truncated Laplace distribution is possible, in which case 



k 



The condition on tt^ in Prior C is satisfied for instance by Poisson distributions. 

The restriction of the prior to Sobolev balls is required to obtain a proper 
concentration rate or even consistency of the posterior of the spectral density / 
itself, which is a necessary step in the proof of our results. This is discussed in 
more detail in section [XT] 
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2.3 Convergence rates and BVM-results under different 
priors 

Assuming a Poisson prior for k, RCL (Theorem 4.2) obtain a near-optimal 
convergence rate for l{f,fo)- In Corollary 13.11 below, we show that the optimal 
rate for I implies that we have at least a suboptimal rate for \d — do\- Whether 
this can be improved to the optimal rate critically depends on the prior on k. 
By our first main result the answer is positive under prior A. The proof is given 
in section m 

Theorem 2.1. Under prior A, the posterior distribution has the asymptotic 
expansion 



n 



(d - do - bnido)) <z\X 



$(z) + op„(l), 



(2.3) 



where, for r^^ ~ ^j>k +i some small enough S > 0, 

^ oo 

bnido)^— Vjeo,j+Yr, + oin-'/^-'kl,/^), K„ = 



j=k„+l 



V2 

y/nrk„ 



■.Zn, 



Zn being a sequence of random variables converging weakly to a Gaussian vari- 
able with mean zero and variance 1. 



20-1 



Corollary 2.1. Under prior A, the convergence rate for d is 5n = {n/\ogn) 
i.e. 

lim [n(d : \d - do\ > 5n\X)] = 0. 

Equation (|2.3p is a Bernstein- von Mises type of result: the posterior distribu- 
tion is asymptotically normal, centered at a point do + bn{do), whose distribution 
is normal with mean do and variance 2/(nrfc„). The expressions for the posterior 
mean and variance give more insight in how the prior for k affects the posterior 
rate for d. The standard deviation of the limiting normal distribution (|2.3p is 

y'2/{nrk„) = 0{n "-f (logn)-"') and bn{do) equals 

oo 

Tl-jOo,-j+OpSkIn^)^o{n-^/^-^'kU^). 

From the definition of fc„ and r^^ and the assumption on it follows that 



1 



-2/3-2 



o{k, 



(2.4) 



l>k„ 



See also (1.9) in the supplement. Hence, when the constant kA in (|2.ip is small 
enough, 

\bn{do)\ < Sn, (2.5) 



8 



and we obtain the (5„-rate of Corollarv r2.ll For smaller k, the standard deviation 
is smaller but the bias bn{do) is larger. In Theorem 12.21 below it is shown that 
this indeed leads to a suboptimal rate. 

An important consequence of the BVM-result is that posterior credible re- 
gions for d (HPD or equal-tails for instance) will also be asymptotic frequentist 
confidence regions. Consider for instance one-sided credible intervals for d de- 
fined by P'^{d < Zn{a)\X) = a, so that Zn{ct) is the a-th quantile of the posterior 
distribution of d. Equation (|2.3p in Theorem 12.11 then implies that 



2k 

z„(a) =do + bnido) + \ —^-\a){l + opjl)). 



As soon as '}2j>k P^^l j " o((logn) we have that 



and 



Zn{a) =do + y/2/{nrkjZn + v'2/(nrfej$-i(a)(l + op„(l)) 



{do < zn{a)) = P (Z„ < + o(l))) = a + o(l). 



Similar computations can be made on equal - tail credible intervals or HPD 
regions for d. 

Note that in this paper we assu me that the smoothness P of /o is greater than 
1 instead of 1/2, as is required in iMoulines and Soulier ( 20031 ). This condition 



is used throughout the proof. Actually had we only assumed that /3 > 3/2, the 
proof of Theorem 12.11 would have been greatly simplified as many technicalities 
in the paper come from controlling terms when 1 < /3 < 3/2. We do not believe 
that it is possible to weaken this constraint to /3 > 1/2 in our setup. 

Our second main result states that if k is increasing at a slower rate than 
km the posterior on d concentrates at a suboptimal rate. The proof is given in 
section [S] 

Theorem 2.2. Given (3 > 5/2, there exists do € 6(/3, Lq) and a constant k^, > 
such that under prior B and C defined above, 

U{\d~-do\ > k,Wn{\ogn)-^\X) ^ 1. 

23-1 J_ 

with Wn = {n/ log n) and = Ci{L ^ Lo)^Hq ■ 

The constant Cw comes from the suboptimal rate for \d — do\ derived in 
Corollary 13.11 Theorem 12.21 is proved by considering the vector 60 defined by 
Oo.j = CQj~^^^^^{\ogj)'^, for j > 2. This vector is close to the boundary of the 
Sobolev-bah e(/3,Lo), in the sense that for ah > 13, = +0°- The 

proof consists in showing that conditionally on fc, the posterior distribution is 
asymptotically normal as in (j2.3|l . with k replacing A:„, and that the posterior 
distribution concentrates on values of k smaller than 0{n^^^^^~^^'^), so that the 
bias bn{do) becomes of order ?i;„(logn)^^. The constraint /3 > 5/2 is used to 
simplify the computations and is not sharp. 
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It is interesting to note that similar to the frequentist approach, a key issue 
is a bias- variance trade-off, which is optimized when k ~ n^/^^^^. This choice 
of k depends on the smoothness parameter j3, and since it is not of the same 
order as the optimal values of k for the loss l{f,f') on the spectral densities, 
the adaptive (near) minima x Bayesian nonparametric procedure proposed in 



Rousseau and Kruiieij (|201lh does not lead to optimal posterior concentration 



rate for d. While it is quite natural to obtain an adaptive (nearly) minimax 
Bayesian procedure under the loss .) by choosing a random fc, obtaining an 
adaptive minimax procedure for d remains an open problem. This dichotomy is 
found in other semi-parametric Bayesi an problems, see for instan ce Arbell ( 2010l ) 
in the case of the white noise model or Rivoirard and Rousseau ( 2010[ ) for BVM 
properties. 



3 Decomposing the posterior for d 

To prove Theorems 12.11 and 12.21 we need to take a closer look at (|1.3p , to un- 
derstand how the integration over 8^ affects the posterior for d. We develop 
9 — >■ ln{d, k, 0) in a point Od.k defined below and decompose the likelihood as 

exp{;„(d, fc, 8)} ^ cxp{/„(fi, k)} cxp{;„(fi, k, 6) - ln{d, k)}, 

where /„(c?, k) is short-hand notation for Z„(d, k, Od.k)- Define 

In{d,k)^ [ e'"('^^'^^'')"'"('^-''^)d7re|,(0), (3.1) 

where 8^ is the generic notation for 8fe(/3 — ^, L) under prior A and Qk{(3, L) 
for priors B and C. The posterior for d given in (|1.3p can be written as 

, s V?l„7rfc(fc) f„e'"(''''=)~'"('^°''=)/„(d,/fc)d7rd(d) , ^ 

n(rf £ D\X) ^ ^ '{° ^ ' ' ^ V (3.2) 

Er=o -^kik) J\l eU<i,k)~iM)i^^d, k)dnd{d) 

The factor exp{/„((i, k) — ln{do, k)} is independent of 0, and will under certain 
conditions dominate the marginal likelihood. In section 13.21 we give a Taylor- 
approximation which, for given fc, allows for a normal approximation to the 
marginal posterior. However, to obtain the convergence rates in Theorems 12.11 
and 12.21 it a-lso needs to be shown that the integrals /„(c?, fe) with respect to 
9 do not vary too much with d. This is the most difficult part of the proof of 
Theorem 12 . 1 1 and the argument is presented in section [3T3l Since Theorem l2.2l is 
essentially a counter-example and it is not aimed to be as general as Theorem 
12.11 as far as the range of /3 is concerned, we can restrict attention to larger /3's, 
i.e. /3 > 5/2, for which controlling /„(d, k) is much easier. 

3.1 Preliminaries 

First we define the point 9d.k hi which we develop 9 — > l„{d,k,9). Since the 
function log(2 — 2cos(x)) has Fourier coefficients against cosjx, j G N equal to 
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0, 2, |, |, • • ■, FEXP-spectral densities can be written as 



|1 - e'^\-^'^ exp <^ J2 % cos(ja;) i - cxp <^ ^(0^ + dry,) cos(jx) 

Given / = fd,k,e a-nd /' = fd',k'.e' we can therefore express the norm l[f, /') in 
terms of {9 -9') and {d ~ d'): 

^ OO 

lif, n = 2 E((^^- - (^'j) + - '^'))', (3.3) 

where 9j and 0^- are understood to be zero when j is larger than k respectively 
k' . Equation p.3p imphes that for given d and k, l{fo,fd,k,e) is minimized by 

oo 

9d,k argmiuggKfc+i ^(6', - 9oj + {d - do)Vjf = ^o[fc] + (4 - c')??!/!;]- 

3=0 

In particular, = 9^^^ minimizes l{fo,fd.k.9) only when d = do! when d ^ do 
we need to add (do — d)r][^ . The following lemma shows that an upper bound 
on l{fo, fd,k,e) leads to upper bounds on \d — do\ and \\9 — 9o\\. 

Lemma 3.1. Suppose that 9 G Qkij, L) and 9o G 0fc(/3, Lo), where j < f3. Also 
suppose that for a sequence an — > 0, l{fo, fd,k,e) < ctn f^''' '^^^ '^^ Then there are 
universal constants Ci, C2 > such that for all n, 

\d-do\ < Ci{L + Lo)^a7^ , P - 9o\\ < C2{L + L^)^ . 

Proof. For all (d, fc, 9) such that l{fd,k,e, fo) < Q^n, we have, using p.3p . 
> 2Z(/d,fc,e, /o) = 2(00,0 - ^o)' + ^ {{Oo,, - 9,) + fj,{do - d)f 



> Y.^0o, - 9,f + {d- dof E - 2|d - dol V] T.(^0o, - 9,Y 

j>i j>i V ^-^ V •'-^ 

= (!l^?-^^o!|-|d-do|||77||)^ 

The inequalities remain true if we replace all sums over j > 1 by sums over j > 
m„, for any nondecreasing sequence m„. Since |j (rjj lj>m )j>i |1 ^ is of order 
and |l(0-0o||,l,>™J,>if < m-27^.^„^Jl+j)2/^(0,.-ij2 < 2{L+Lo)m~^\ 
_ j_ 

setting TO„ ~ ctn ' gives the desired rate for |d — do| as well as for ||6' — 6'oj| . □ 
The convergence rate for Z(/ o, fd,k,e) required in Lemma 13.11 can be found in 



Rousseau and Kruiieii (|201lf ). For easy reference we restate it here. Compared 



to a similar result in RCL, the log n factor is improved. 



11 



Lemma 3.2. Under prior A, there exists a constant Iq depending only on Lg 
and kA (and not on L) such that 

n{{d,k,e):l{Uk,eJo)>llSl\X)^0, 

where 6n = (71/ log n) -"f . Under priors B and C, this statement holds with 
€n — {n/\ogn) replacing (5„. 

In the proof of Theorem l2.1l (resp. 12. 2p . this resuh aUows us to restrict attention 
to the set of spectral densities / such that l{f,fo) < ^qS^ (resp. /ge^). In 
addition, by combination with Lemma l3.1l we can now deduce bounds on jd — (io| 
and \\d — 0d,k\\- These bounds, although suboptimal, will be important in the 
sequel for obtaining the near-optimal rate in Theorem 12. II 

Corollary 3.1. Under the result of Lemma and prior A, we can apply 
Lemma \3.1\ with a\ = IqS'^ and 7 = /3 — i, and obtain 

Udid:\d~do\>vn\x)^o, n{\\e-ed,k\\ > 2?o<5„|x) ^ o, 

where w„ = Ci{L + Lo)^i^-'^ll'^ ^(n/logn) ^/s _ Under priors B and C we 

have 7 = /3; the rate for \d — do\ is then Wn = Cu,(n/logn) and the rate 

1 

for \\0— dd,k\\ 2/oen- The constant Cw = Ci{L + Lq)^ Iq^^ is as in Theorem 

Proof. The rate for |c? — do| follows directly from Lemma [3.11 To obtain the 
rate for \\6 — Od,k\\, let a„ denote either IqSu (the rate for l{fo,f) under prior 
A) or loen (the rate under priors B and C). Although Lemma [3T] suggests that 
the Euclidean distance from 9o to 9 (contained in 0fe(/3,L) or 8fc(/3 — ■^,L)) 
may be larger than q;„, the distance from 9 to 9d,k is certainly of order q;„. To 
see this, note that Lemma |3^ implies the existence of d, k, 9 in the model with 
Kfo, fd,k,e) < a\. From the definition of 9d,k it follows that /(/„, fd,k,ea.J ^ "n- 
The triangle inequality gives \\9 - 9d.k\\'^ = Kfd,k,0,fd,k,Sa,J ^ ^a^. □ 

The rates Vn and Wn obtained in Corollary [3T] are clearly suboptimal; their 
importance however lies in the fact that they narrow down the set for which 
we need to prove Theorems 12.11 and 12.21 To prove Theorem 12.21 for example it 
suffices to show that the posterior mass on fc„zi;„(log < jd— < Wn tends 
to zero. Note that the lower and the upper bound differ only by a factor (logn). 
Hence under priors B and C, the combination of Corollarv 13 . 1 1 and Theorem l2.2l 
characterizes the posterior concentration rate (up to a log n term) for the given 
9o- Another consequence of Corollarv 13. II is that we may neglect the posterior 
mass on all {d, k,9) for which \\9 — 9d,k\\ is larger than 2lo6n (under prior A) or 
2Zoen (under priors B and C). 

We conclude this section with a result on 9d.k and 6fe(/3, L). In the definition 
of 9d,k we minimize over R*''+^. whereas the support of priors A-C is the Sobolev 



12 



ball 8fc(/3, L) or Ofc(/3 — h^L). Under the assumptions of Theorems 12.11 and 12.21 
however, Od,k is contained in 8fe(/3 — ^, L) respectively Qk{P, L). Also the Z2-ball 
of radius 2Zo<5n (or 2Zoe„) is contained in these Sobolev-balls. 

Lemma 3.3. Under the assumptions of Theorem \2.1\ Bk{0d,k,'^loSn) "is con- 
tained in Qk{(3 — ^, L) for all d £ [do — Vn,do + Vn], if L is large enough. In 
particular, Bd.k & Qk{f3 — ■^,L)- Similarly, under the assumptions of Theorem 
\MM Bu{9d,kMoin) C efc(/3,L), for all d € [do - + w„] . 



Proof. Since the constant /q is independent oi L, 6 G Bk{0d,k,'2la5n) implies 
that for n large enough , 

k k k 

Y.e]{j + ifP-' <2j2{e^ Bd.k)]ij + ir-' + 2Y,{ed,k)]{3 + if-' 

j=0 j=0 j=0 

< 8(52 (Lo)(n/ log n)'fe^(fc„ ^ ^)2/3-i _^ 4 g ^2^^ ^ ^^2,3-1 

j=o 

+ 16(d-dofY^f^~'. 
The first two terms on the right only depend on Lq, and are smaller than L/4 

1 _/3-l 

when i is chosen sufficiently large. Because u„ = Ci{L+Lo)'^f~^ Iq (71/logn) , 
the last term in the preceding display is at most 

CfiL + Lo)^ if^' {n/ log n)-^ kf~\n/ log n) ^ , 

which, since /3 > 1, is smaller than L/2 when L is large enough. We conclude 
that Bk{9d,k, 2loSn) is contained in 9fe(/3 — |, £) provided L is chosen sufficiently 
large. The second statement can be proved similarly. □ 



3.2 A Taylor approximation for ln{d,k) 

Provided that the integrals /„ (d, k) have negligible impact on the posterior for 
d, the conditional distribution of d given k will only depend on exp{Z„(d, fc) — 
ln{do,k)}. Let ln\d,k)^ l^\d,k) denote the first two derivatives of the map 
d I—)- /„(d, k). There exists a d between d and do such that 

Z„(d,fc) = ln{do, k) + id- do)li'^ (do, fc) + ^'^ ^^"^^ (d, k). (3.4) 
Defining 

6„(d) = -^|i^, 

l^n\d,k) 



13 



which is the 6„ used in Theorem 12.11 we can rewrite p.4p as 

/„(d, k) - l^do, k) = Vo,fc))' ^ ^(^2)(J^ ^) (d - do - 6„(J))' . (3.5) 

2 d^(d,fc) 2 

Note that each derivative ln\d, k), i = 1, 2, can be decomposed into a centered 
quadratic form denoted S{ln\d, k)) and a deterministic term V{ln\d, k)). In 
the fohowing lemma we give expressions for ln\do, k), l^\d, k) and b„, making 
expUcit their dependence on k and Oq. Since fc„ < fc„ and w„ < u„ (see CoroUary 
13. ip the result is valid for all priors under consideration. The proof is given in 
appendix Rl 

Lemma 3.4. Given (3 > 1, let do G Lq). If k < fc„ and \d— do\ < Vn, then 
there exists 6i > such that 

li'Hdo,k) 5(/W(do,fc))+I?(/W(d„,fc)) 

OO 

= 5(/W(d<„fc)) + 5 eo,r;,+o(n'(fc-^+3/2^,^-i/(2,)))^ 



j=k+i 

/ 1.1/2 i.-2l3 + l+e\ 1 

eHd,k) = /(2)(d„fc)(^l + --j^ + j =--nr,(l + Op„(n-^0), 

where S{ln\do, k)) is a centered quadratic form with variance 

Var{S{l^:\do,k))) = |^r,,^(l + o(l)) = ^(l + o(l)) = 0{nk-^). 

Consequently, 

in (Cl, K) fc 



with 



^25(^^^)(d.,fe))(l+op.(.~^))^^^ (.^-fc-/^-/^ + .-) 
nrfe 

25(d'^(do,fc)) , 

^ ^ = Op (n 2fc2). 



(3.6) 



Remark 3.1. Recall from dM]) that ^ E^fc+i ^'oj% 0(fc-''+i/2). T/ie 

term 2S{ln\do,k)) / {nrk) is Op^{k^^^^/^) whenever k ~ n^/^^^\ which is the 
case under all priors under consideration. 

Substituting the above results on ln \ In^ and 6„ in (|3.5p . we can give the 
following informal argument leading to Theorems 12.11 and Theorem 12.21 If we 
consider k to be fixed and /„ (d, fc) constant in d, then p.Sp implies that the 
posterior distribution for d is asymptotically normal with mean do + bn{do) and 
variance of order k/n. 
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3.3 Integration of the short memory parameter 

A key ingredient in the proofs of both Theorems 12 . 1 1 and 12 . 2 1 is the control of the 
integral k) appearing in (|1.3p . whose dependence on d should be negligible 
with respect to exp{Z„(d, k) — /„(do7 k)}. In Lemma 13.51 below we prove this to 
be the case under the assumptions of Theorems 12.11 and 12.21 For the case of 
Theorem l2 . 2l this is fairly simple: the conditional posterior distribution of 6 given 
(d, k) can be proved to be asymptotically Gaussian by a Laplace-approximation. 
For smaller (3 and larger k the control is technically more demanding. In both 
cases the proof is based on the following Taylor expansion of k, 9) around 



J 



id-Od,k)'^''>WL{d,k) 



Lid, fc, 9) - Ud, k) = ^ ^ '^^^ + Rj+uM, (3.7) 



h,...,lj=0 
k 



where 

dHn{d,k,9d,k) 
d9i,...d9i^ ' 

'i j+i=0 ' ^ 

(3.8) 

The above expressions are used to derive the following lemma, which gives 
control of the term /„(d, k). 

Lemma 3.5. Under the conditions of Theorem \2.1[ the integral In{d, k) defined 
in (|3.ip equals 

' \d - do\n^-^^\ J n2"^"''' 



/„((io, A:)cxp < op^(l) + op^ I ^= ) + op^ ( {d - do) 



for some 82 > 0- Under the conditions of Theorem[ 

I„{d,k) =:/„(do,fc)cxp{op„(l)}. 

The proof is given in Appendix [Cl and relies on the expressions for the 
derivatives V-' /n given in Appendix iBl Lemma |3 . 5 1 should be seen in relation to 
Lemma 13.41 and the expressions for n(rf|Ar) and Z„(d, k) — ln{do, k) in equations 
(|3.2p and p.4p . Lemma 13.51 then shows that the dependence on the integrals 
In{d, k) on d is asymptotically negligible with respect to ln{d, k) — ln{do, k). This 
is made rigorous in the following section. 

4 Proof of Theorem 12.11 



By Lemma 13.21 we may assume posterior convergence of l{fo, fd.k.e) a-t rate IgSf^, 
and, by Corollarv l3.11 also convergence of jd — doj at rate u„. By Lemma [ 
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we may restrict the integration over 9 to Bk{Od,k,'2loSn)- Let r„(2) = {d : 
y/^^{d — do — bn{do)) < z}. Under prior A, it suffices to show that for k = kn, 



/i 



r„(^)' 



J„{d,k)-l,^{d„,k) 



{Sd,k,2loSr,) 



J^{d,k,e)-i^(d,k)dT:g^^{e)dT:d{d) 



Dn ■ /|,_,„|<,„ /s,(,,.,,2W„) e^-^<^^'^fi)-^^^'^-'^)d^e\k{e)d^M 

exp{/„(d, k) ~ ln{do, k) + log/„(d, k)}dTTd{d) 



\d~do\<v. 



cxp{l„{d,k) - l„{do,k) + \ogI„{d,k)}dTrd{d) 



$(z)+Op„(l). 

(4.1) 



Using the results for k) — ln{do, k) and k) given by Lemmas 13.41 and 
[531 we show that for A„ C M" defined below such that Po{A„) 1, 



^ <<I>(;^) + o(l), ^>$(z) + o(l), VXgA„. 



(4.2) 



Since P"(A„) ^ 1 this implies the last equality in ()4.ip . 

Note that Lemmas 13.41 and 13.51 also hold for all S'^ < 6i and S2 < S2. In the 
remainder of the proof, let < 5 < mm{di, 62)- For notational simplicity, let 
V ~ 'D{lii\do, k), the deterministic part of ln\do,k). For a sufficiently large 
constant Ci and arbitrary ei > 0, let An be the set of X G K" such that 



|log/„(d,fc) -log/„(do,fc)| < ei + {d-do)^k 



+ \d — do\k 2 n' 



i^\do,k)~V 



< Cl??2 fc 2 y/logn, 



ll?\d,k) + hirk 



< n 



for all |c? — dol < w„. Since k ^ kn and (3 > 1, Lemmas 13.41 and 13.51 implv 
that P"(Afj) — 0. We prove the first inequality in (|4.2p : the second one can be 
obtained in the same way. Using p.4p and the definition of An, it follows that 
for all X e An, 



ln{d, k) - ln{do,k) + log/„(d, fc) - \ogIn{do,k) < ei + (d- dof 



+ \d- dol^ + {d- doWHdo, k)-^{d- do)2(l - n-') 

fc2 4 

' ^ 2/W(do,fc) V 



-5 



< 2ei - 


nrk 
4 


0- 


2 


< 3ei - 


nrk 
4 




2 



-.5 



{li'\do,k)y 
(1 - ;4) nrfc 



1 



nrk ^ 
2 



|d-do| 



fc2 



d — dn 



d — 



bn{do,k) 



1 - 



^^(do, fc) 

^-^ ) 

^ {liP{do,k)r 

(1 - nrk ' 



(4.3) 



The third inequality follows from (|2.5p and Remark 13.11 by which bn{do) — 
0{k~^~^i) ~ 0{Sn)- This implies that \bn{do)\k^ini^^ < ei, again for large 
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enough n. Similar to the preceding display, we have the lower-bound 
l„{d, k) - l„{do,k) + log/„(d, k) - log/„(do,fc) 



> -3ei 



{l + 2n-^) \ d-do 



K{do,k) 
(l + 2n-'5) 



d — dn 



bn{do, k) 



Note that 



exp ' 



{l^^\do,k)f {l^n\do,k)f 



(l-2n-^)nrk {1 + 2n-^)nrk 



(1 + 2n^^)nrk 



cxp{o(l)}, 



(4.4) 



(4.5) 



which follows from the expression for ln\do, k) in Lemma [3^31 the definition of 
An and the assumption that X G A„. Therefore, substituting (|4.3p in A^„ and 

(|4.4p in Dn, the terms ^nrl'^'^ cancel out and by (|4.5p we can neglect the 
difference between nlS'^l'sl'^ and Iin^-(fki^il]_ 

To conclude the proof that Nn/ Dn < $(z) + o(l) for each X € An , we make 
the change of variables 



1 ± 2n-' 



where we take + in the lower bound for Dn and — in the upper-bound for 
Nn- Using once more that 6„(do) = 0((5„), we find that for large enough n, 
I'^l ^ ^y/nrk implies |d — do| < u„. Hence we may integrate over \u\ < ^^/nfk 
in the lower-bound for £)„. In the upper-bound for Nn we may integrate over 
u < z + ei. 

Combining (|4.3p - (|4.5p . it follows that for all ei and all X e An, 



Nn 
Dn 



< e 



7ei 



1 + 2n- 



1 - 2n-^ 
, exp{ 



/»<z+eiCxp{-^u^ + Cn ^\u\}du 
L|<%./TJFi:Cxp{-|u2 - Cn-^\u\}du 



+ Cn-^\u\}d 



nrk ^ 2 



flu 



cxp{—^ — Cn 



Similarly we prove that for all ei, Nn/Dn > ^{z — ei)e ^'^^ when n is large 
enough, which terminates the proof of Theorem 12. II 



5 Proof of Theorem [23] 

Let (3 > 5/2 and 9oj = coj^^^^^^logj)^^ . When the constant cq is chosen 
small enough, 9o € 0(/3, Lq). In view of Corollarv l3.1i the posterior mass on the 
events {{d,k,9) : |16' — ^d,fc|| > 2Zoen} and {{d,k,d) : \d — do\ > Wn} tends to zero 
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in probability, and may be neglected. Moreover Lemma 13.11 implies that with 
posterior probability going to 1, — 6'o|| ^ (n/logn)~('^^^/^'/(^'^+^). However, 
within the (fc + l)-dimensional FEXP-model, \\9 — do\\ is minimized by setting 
6j ~ 9oj {j ~ 0, . ■ . , k), and for this choice of 6 we have 

l>k 

Consequently, the fact that ||6' - 6*011 < (n/ log n)-('^-i/2)/(2/3+i) implies that 
k > k'^ ■= fci(n/logn)(/^-i/2)/('3(2/3+i))(iogn)-i//3^ for some constant h. We 
conclude that 

n(fc<fc::ix) = opji), 

and we can restrict our attention to fc > fc". 

We decompose Ild{\d — do\ < fct,u'„(log ri)"^, fc > k'^\X) as 



J2 n{\d-do\<k^Wn{\ogn)-\k = m\X) 

Tl{k ^ m\X)Urn{\d - do\ < k^Wn{logn)-^\X), 



where n^dd— do\ < fct,'u;„(logn)^^|X) is the posterior for d within the FEXP- 
model of dimension m + 1, i.e. Ilr,i{\d—do\ < kj;Wn{iogn)~^\X) -.— YLdd—dol < 
k^Wnilog n)"-"^ |fc = TO, X). 

To prove Theorem 12.21 it now suffices to show that 

Il{k^m\X)=n{k'^<k<k'„\X)^l, (5.1) 
E^Ilk{\d-do\<k,w„{\ogn)-^\X)^0, < k < k^ (5.2) 



The c onvergence in (|5.ip is a by-product of Theorem 1 in Rousseau and Kruiieil 
(j2011l ). In the remainder we prove (j5.2p . For every fc < fc„ we can write, using 
the notation of (|4.ip . 



Ilki\d~do\ < fc„u;„(logn)-i|X) < 



Nn,, 



D. 



n,k 



/|d-do|<fc„«>„(iogn)-i exp{Z„(d,fc) - lnido,k) + log In{d,k)}dTrdid) (^■^) 
I\d-d \<w exp{ln{d,k) - l„{do,k) + log In{d,k)}dTTd{d) 



Let ^2 > and An be the set of X S M" such that 

|log /„((/, fc) - logIn{do,k)\ < ei, 
{do, k) - (do, fc)) I < n5 fc- ^ VbfTI, 

l'i\d,k)^V{e\doM < ei,^-(2+*2)/(2/J+i) 
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for all \d — do\ < Wn and k'^ < k < k[^. Compared to the definition of An 
in the proof of Theorem 12. 1[ the constraints on /„ {d,k) and /„ are different. 
For the latter, recall from Lemma [331 that log/„((i, k) = \ogln{do, k) + op^(l), 
uniformly over d e (do — Wn, do + Wn)- As in the proof of Theorem 12.11 it now 
follows from Lemmas 13.41 and 13.51 that ) — > 0. We can write 



<p:{ai)+e^^ 



Nn 



and bound Nn.k/ Dn,k pointwise for X G An- Since when k G (fc^, fc„), 
{liPido,k))^ 



2\ll?Hdo,k)\ 



-(2+52)/(2;3+l) 



o(l) 



on An, for all 62 > 0, analogous to (|4.3p and (|4.4p . we find that for all X G An, 
by definition of bn{do), 

ln{d, k) - ln{do, k) + log/„(d, k) < 2e, - {d-do- hn{do)f ' 



ln{d,k) - ln{do,k) +logJ„(d, fc) > -2ei 



\l^n\do,k)\ 



{d - do - bn{do)) + 



2\l[?{do,k)\ 

2 , {l^n\do,k)Y 



2\l^/^\do,k)\ 



when n is large enough since k > fc^'. We now lower-bound bn{do) by bounding 
the terms on the right in p.6|) in Lemma 13.41 By construction of 60 it follows 
that 

^k'T.^^'^",, = cor^T^ ^r^-V(logj) > ck-^+Hiogk)-\ 

j>k j>k 

for some c > 0. Since X € An, 2S{ln\do,k)) / {nrk) < 2 ■\/A?7?^^/logn. Since 
k < k'n, this bound is o(fc~'^+5 (log k)~^). The last term in (|3.6p is o{n'^~^) when 
/3 > 5/2, and hence this term is also o(fc~'^~ 2 (log fc)~^). Therefore, the last two 
terms in p.6p are negligible with respect to r-^T^ 'l2j>k J^^^o.j- We deduce that 
bn{do) > ck-^+i {log k)-' > cn-(2/J-i)/(4/3+2)(iogn)-(2/^+3)/(4^*+2) for n large 
enough. 

Consequently, when the constant k^, is chosen sufficiently small, y/nfk^{bn{do)- 
kyWn{\ogn)-^) > (c - fc„)7ll/(4/3+2)(log,^)-(/3+l)/(2/3+l) 2„ ^ oo. We now 
substitute the above bounds on ln{d,k) — ln{do,k) + \ogIn{d,k) in the right 
hand side of (|5.3p . make the change of variables u = d — do — bn{do) and obtain 



Nn 



Dn 



fu 



<. — k^Wn (log n)~^ —bn{do) 



exp{-^^}dv 



I\u 



\u\<w„/2 



exp{- 



-}du 



< e 



5ei 



I\v\ 



'lv\<w„y'nrk/8 

This achieves the proof of Theorem 



Iv>z„^M-^}dv 
3xp{ — 



= op„(l). 
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6 Conclusion 



In this paper we have derived conditions leading to a BVM type of resuh for 
the long memory parameter d £ ("515) of a stationary Gaussian process, for 
the class of FEXP-priors. To our knowledge such a result has not been obtained 
before. The result implies in particular that asymptotically credible intervals 
for d have good frequentist coverage. 

A by-product of our results is that the most natural prior (Prior C) from 
a Bayesian perspective, which is also the prior leading to adaptive minimax 
rates under the loss function I on /, leads to sub-optimal estimators in terms 
of d. Prior A leads to optimal estimators for d however it is not adaptive. An 
interesting direction for future work would be to define an adaptive- minimax 
estimation procedure for d. 

More broadly speaking, the approach considered here to derive the asymp- 
totic posterior distribution of a finite dimensional parameter of interest in a semi- 
parametric problems could be used in othe r non - regular mode ls, hence complet- 
ing (ii ot exhaustively) the recent works of Castillo ( 20101 ) and Bickel and KleiinI 
I 2010h . 
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A Proof of Lemma 3.4 

We decompose the first derivative of /„(d, fc) as ln\d,k) = S{ln\d,k)) + 
V{ln\d, fe)), S{ln\d, k)) being a centered quadratic form and T>{ln\d, k)) the 
remaining deterministic term. To simplify notations, in this proof we write 
S = S{ln\do,k)) and V = V{ln\do, k)). Using (1.6) (supplement) and defin- 
ing A = r-i(/d„,fc)T„(i/fc/d„,fc)r-i(/d„,fc), we find that 

2? = -^tr [(T„(/<i„,fc) - Tn{fo))A] , 5 = i {x'AX - tr [T„(/o)A]) . 
From (1.4) and (1.8) in the supplement it follows that 

fo - fd^M = fdMe^'-" - 1) = (Ad„,fc + ie«Al^,)/rf„,fc 

= /d„,fcO(fc-^+i/2), ee(0,(Arf„,fc)+)). 

Consequently, we have 

V = itr [T„(/d„,fe(Ad„,fc + 0{Al^k)))T~\fd„,k)T„{Hkfd^,k)T-H.fd^,k)] 



n 
An 



Hkix){Ad,,kix) + 0{Al^ k{x)))dx + error 



n 



— 7r 
oo 

rijOoj + 0{nk~'^^^^) + error. 
j=k+i 

The last equality follows from (1.9) and (1.11) in the supplement. We bound 
the error term using Lemma 2.4 (supplement) applied to Hkfda,k and fdo,k, 
whose Lipschitz constants are bounded by 0{k) and 0(A:('^/^~^)+ , respectively 
(see Lemma 3.1 in the supplement). Using that ||A(i^ fe||oo = 0(fc~''+^/^) (see 
(1.8) in the the supplement) we then find that the error is 0{k^^^'^^n'^). 

1 - - 2 

The term 5 is a centered quadratic form with variance ^\Tn {fo)ATn {fo)\ ■ 
Applying once more (jA.l[) . we find that 

tr [{Tr,{fo)Af] = tr [{T;^\fd.,k)Tn{HUd^M)f] (1 + 0{\\Aa.^k\\^)) 

= ^ I Hl{x)dx + 0(n'k + |lAd„.fc|U) = nruil + o(n-*)), 

where the term n'^k comes from Lemma 2.4 in the supplement, associated to 
/do,fc and fdo.kHk. This proves the first equality in Lemma 13.41 
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Similar to the decomposition of ln\do, k), we decompose the second deriva- 
tive as ll?\d,k) =V{l^n\d,k)) - 2Si{l^n\d,k))+S2{l^n\d,k)), where 

Slid) = X'Ai^dX - tr[r„(/o)Ai,d], 52(d) = X'A2mX - tr[T„(/o)A2,d], 



(Tnifd.k) — Tn{fda.k)) ( — 2^"^ '^ 



+ tr 



-A 



2,d 



(A.2) 

^l,d = ^{.fd,k){Tn{Hkfd,k)Tn ^{Jd,k))'^ , ^2,d = T„ [f d,k)Tn{Hl f d.k)Tn ^{fd,k)- 

To control 'D2{d) we use a first order Taylor expansion around do, implying 
that V2{d) = V2{do)+O{\d^do) sup|^,_^j<,g^ |2?2('^')l- First we study 2?2(do)- 
At d = do, the right-hand side of (|A.2p equals 



— / i?^2(a;) (1 + (e'^''-'" - 1)) dx + 0{kn') 



-nrk 



l + 0(fc-'5+i/2 + fc2/„i-e) 



(A.3) 



The 0{krf) term is obtained from Lemma 2.4 (supplement), applied to f2j = 
Hkfda,k and f2j-i = fdo,k, with Lipschitz constants 0{k) for the former and 
0(^(3/2-/3)+-) f-Qj. ^jjg latter, together with the bound ||Arf„,fc||oo = 0{k-^+'^/'^). 
Using 



(A.4) 



^i,d — ^3 (r„ ^{fd,k)Tn{Hkfd,k)) T^i ^{fd,k) 

+ '^T^^{.fd,k)Tn{HlfdM)Tn^{fd,k)Tn{Hkfd,k)T^^{fd,k) 

and a similar expression for the derivative of d — > A2.d, it follows that 
< tr [(T„(/rf„,,)r-i(/,,,,) H-/„)(T„(|i7,|/,,,,)T-i(/,,^,))3] 
+ tr [{TMd^,k)T-\fd',k) + In)iTn{\Hk\fd'^k)T-\fd',k))TniHlfd',k)T-\fd',k)] 
+ tr [{TMd^,km^\fd',k) + In)Tn{\Hkffd'^km-;\fd'^k)] 

We control the first term of the right hand side of the above inequality, the 
second and third terms are controlled similarly. Note first that 

tr [r„(/d„,fe)T-i(/rf,,fe)(T„(|i7fc|/d-,fe)r,7^(/d',fe))'] 
= |7;?(/d„,fe)r-i(/d,,fc)T„(|iJfe|/d,,fc)T-i(/d,,fe)Ti(|i7fe|/d.,fc)|' 

< \\Thfd.,k)TnHfd',kW 



X \\Tnmd',k)T^i\Hk\fd',k)f\Tn Hfd'M)Tni\Hk\fd'.k)Tn Hfd',k)\' 



< 



n'\Tn Hfd',k)Tni\Hk\fd',k)Tn Hfd',k)\' 



(A.5) 
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where the last inequahty comes from Lemma 2.3 in the supplement. Note also 
that 

kl"'"' < fd'ix) < \x\-^'' and Tr,{\Hk\U,k) < Tr^UHkllxr'"' ), T^{\fd',k) > TM^'''') 
and replace fd' by in (jA.5[) . then 

|T-i/2(/,.,fe)T„(|i7,|/,,fe)T-V2(y^, < g + 

using Lemma 2.4 in the supplement associated to jiJ^j jccj^^'' which has Lipschitz 
constant k. This leads to 'D2{d') = O {n'^j) , which implies that for all /3 > 1, 

V2{d)^V2{do)+o{\d~doW+^k-^) = -"^{l + oin-')). 

For the stochastic terms in /„ {d, k) we need a chaining argument to control 
the supremum over d e {do — Vmdo + Vn)- We show that for all e' > and 

sup |5i(d)|>7n) = 0(1), (A.6) 

y|d-do|<B„ J 

i.e. that Si{d) = Op^(7„). The same can be shown for 52 (rf) using exactly the 
same arguments. Consider a covering of (do — Vn , do + v^) by balls of radius 
centered at d^. j = 1, . . . , J„ with J„ < 2u„n. Then 

sup |5i(o?)| < max |5i(o?j)| + sup \Si{d) — Si{d)\, 

\d-d^\<v„ 3 |d-rf'|<n-i 

and 



p: ( sup |5i [d] I > 7n ) < p: \ sup |5i id) -S,{d')\> 

y|d-do|<C„ J y|d-d'|<n-i ^ ^ 

1 



J„ max |5iK)|>-7. 
i<j<,/,i \ ^ 



(A.7) 



To control the first term on the right in (|A.7|) . note that for a standard normal 
vector Z and some d* G {d,d'), 



Slid) - Slid') = id- d') i^Z*T,Hfo)A[^a,T^{fo)Z - tr [Tnifo)A[^dA) > 

with A'^ ^ as in (|A.4p . Using Lemma 2.3 (supplement) and the fact that || < 
||A||||B|| for all matrices A and B, it follows that \\T^ {fo)A[^^,T^ ifo)\\ = O(n'), 
and hence Z'T^ ifo)A[ ,,,T^ {fo)Z < Z'Z\\tI ifo)A[^a.T,Hfo)\\ = 0(n^)Z*Z. 
Similarly, it follows that tr T'ri(/o)^i d* ^ Consequently, when e = e'/2 we 
have 

\Si{d) - Siid')\ < n-^ {Z*Zn' + n) , 
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uniformly over all d^d! such that \d — d'\<n ^. Since 1 = o(7„), 

P: ( sup \Si{d) - Si{d!)\ > ^7,, ) <P{Z'Z > ni-^7n/4) = o(l). 

\\d-d'\<n-i ^ J 

To bound the last term in (jA.7|) . we apply Lemma 1.3 (supplement) to 
{Z*AZ - tr[A])|Ap\ with A = (/o)Ai,<jTi (/„) since as seen previously 
\AY = 0{n/k) = 0(7271-2") for a small enough, it follows that 

Since J„ increases only polynomially with n, this finishes the proof of (jA.6p . 



B Control of the derivatives in 9 on the log-likehhood 

Before stating Lemma IB. II we first give a general expression for the derivatives 
of /„(d, k, 9) with respect to 0. For all j >1 and Z = (/i, . . . , Ij) S {0, 1, . . . , fc}^, 
let a = ((7(1), • . • ,(T(|cr|)) be a partition of {1, . . . , 7}. Let \a\ be the number of 
subsets in this partition and the ith subset of {1, . . . , j} in the partition a. 
Denoting the vector {lt,t S cr(i)), we can write 

For notational ease we write a(^i)fd,k,e ■= Vi^^,, fd,k,9- The derivative Qg'"^^gef ^ 
can now be written in terms of the matrices 

kl 

Baid,9) = []B,(,)(d,0), B,(,)(d,0) = T„(V,(,)/d,fc,e)T-i(/d,fc,e). (B.l) 

i=l 

There exist constants ba,Ca and d^ such that 

a^/n(rf,fc,g) 

= Y.^- {X'T-'iUk,e)BAd,0)X - tr [TMo)T~'ifd,k,e)BAd,e)]) 3) 
+ ^ c^tr [B„id,e)] + J2 d.tr [{T^{fo)T~\fd,k,e) ~ /„)B.(d,e)] , 

where Sj is the set of partitions of {l,...,j}. For the first two derivatives 
(j = 1, 2) the values of the constants b^, Ca and da are given below in Lemmas 
IB. 41 and IB. 51 For the higher order derivatives these values are not important 
for our purpose; we will only need that for any J > 1, the constant Ca is zero if 

H = i. 



25 



The following lemma states that ln{d, k, 9) ~ ln{d, k) is the sum of a Taylor- 
approximation J2j=i ^^"^'^'''^ ^.^ in.{do,k) terms whose dependence on d can 
be negligible. Since the proof is involved, some of the technical details are 
treated in Lemmas IB.2I and IB.3I 

Lemma B.l. Given (3 > 1, let k < kn and let d and 9 be such that l{ fo, fd.k.e) < 
l^Sf^. Then there exists an integer J and a constant e > such that uniformly 
over d€ {do - Vn, do + Vn) and 9 € Bk{9dM-, S/o'Jn), 

Ud,K9)-Ud,k).t^l^^^^ 

J ^' (B.3) 

+ {d- do) -^9nA^ - Sd,k) + 5„(d), 

where, for u ~ 9 ~ 9d.k, 

k 

li,...,lj—0 ^^^j 



kl / 

i=l \l<i 



k X 



\Tn{^ a(i)fda,kHk) — Tn{Hkfda,k)Tn ^ {f d„,k)Tn{^ a(i) I da,k)\ X 
Tn^ifda.k) {^\YTn{V a(i)fd„^k)Tn^{fda,k)^ , 

T2Ado,k) = -TAHkfd„M)Tn\fd.,k)BMo,Od^,k). 
and Sn (d) denotes any term of order 

S„{d) = op„(l) + op„ ( \d-do\n^-' \ + ( _ dof^) . (B.5) 



Vk 

When /3 > 5/2 and k < k'„, we can choose J = 2, and (jB.3P simplifies to 

7 M m / M z.^ ^ i9~9d,k)^i^VHAdo,k) , 

Z„(d, fc, 6*) - fc) = 2^ ^ h op^ (1). (B.6) 



Proo/. Recall that by ([STT]) . 

/n(d, fc, 9) - /„(d, k) ^2_^ 

+ ^ \- Kj+i,d{9). 
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To prove (|B.3|) we first show that, writing u = 9 — 6d.k 



^ U<-3^yH Ud,k)-L{do,k)) 



J ^ k 

3 = 1 •'' h,...,lj=0 
J 



V d6i,...dei^ 
{d~do)Y,\gn,M) + 0{Sn{d)). 



d9i, . . . d9i 



(B.; 



J'- 



This result is combined with (jB.7p and Lemma lB.31 below, by which gn,i{u) 
0{Sn{d)). It then follows that ln{d,k,9) — ln{d,k) equals 



E 



' 1 

(d - do) ^ + Rj+iA^) + 0{Sn{dj). 

J=2 



The final step is to prove that Rj+i^d) is op^(l) and hence 0{Sn{d)); to this 
end J needs to be sufficiently large. 

First we prove (jB.Sp . For the factors ^ln{d.,k,9d.k) — ^ln{d,k,9d^^k) we 
substitute (|B.2[) . In Lemma fB . 2 1 below we give expressions for each of the terms 
therein, which we substitute in (|B.8[) . The main terms are {d — do)tr[Ti a-{do, k)] 
and {d - do)tT[T2,cr{do, k)] in (|G.2|) and (|G.3p . which after substitution in (|B.8|) 
give the term {d ~ do) yr5"-j (^) '^^ right. The other terms in (jG.ip - 

(jG.Sp that enter (jB.Sp through (|B.2p are 0(S'„(d)). This is due to the summation 
over , . . . , uij in (|B.8p . and the Cauchy-Schwarz inequality by which 



E 



h,....li=0 



< {Vk\\u\\y < {2laVk5,,y = o(n-^). 



(B.9) 



for some (5 > 0, as < 2Zo(5„ and (|B.8[) is proved. We now control Rj+iA9). 
Combining p.8p and the first inequality in (|B.9[) . we obtain 



\Rj+i,dm < 



:{VkSny+^ 



(J + l)!~ " 



max sup 



d 



J+i; 



,Xd,k,9) 



39, 



■d9i,,^. 



We give a direct bound on this derivative using (jB.2p . For all partitions a of 
{h, . . . ,lj+i} and ah {h, . . . ,l,j+i) e {1, . . . , k}-'+^ , we bound \\B„(^i){d,9)\\, 

using < \\T,Uy^^,^fd,k,e)Tnhfd.k.9)f (see dUD). We bound 

1 _i 

\\Tn {'Va{i)fdMM)Tn ^ {fdM.e)\\ by application of Lemma 2.3 (supplement) with 
/ = fd,k,e and g ~ a(i)fd.k.e- The constant M in this lemma is bounded by 



Ei^.i<Ei(^"^.fc).i+Ei^.- 

i=o j=o j=o 



Td,k 



)j \ < 2^/1+ Vk\\ 



0(1), 



27 



since J2i=o l(^d,fc)i| < 2-\/Z (by Lemma [53]) and H^ — 0^,^11 < (5„. Consequently, 
Lemma 2.3 (supplement) implies that 

\\B,i^){d,e)\\ < K, (B.IO) 

where K depends only on L,Lo and not on n, d nor 9. From the relations in 
and the definition of Bcr it follows that for any cr, d, 9, 

\X'B„{d,9)X\ < X'T-Hfo)XK^''^\\Thfo)TnHfa^k)\\^ < X*T-\fo)XK^''^n\ 

|tr [B^{d,9)]\<nK\''\\\Tl{fo)T--HUk)f<n^+'K\"\. 
Therefore we have the bound 

\Rj+i{d,9)\ < CK''+\,^{Vk\\9-9dM\y^' {X'T-\fo)X + n) . (B.ll) 

Since k < fc„, - ^^,^11 < 6n and the term X^T-^{fo)X in ((BJT|) is the sum 
of n independent standard normal variables, there is a constant c > such that 

Pol sup sup \Rj+i{d,9)\>n-A <e-"\ 

\\d-d,\<v„ \\e-9d,k\\<2loS„ J 

provided we choose J such that (J + 1)(1 — 1//3) > 2. This concludes the proof 
of (lR3l) . 

To prove (jB.6|) we first show that for J = 2, \Rj+i{d, 9)\ = op^(l). Since k < 
fc;, l3 > 5/2 and ||6'-^d,fc|| < 2/oe„, we can choose J+1 = 3 > (2^ + l)/(^- i), 
and the preceding inequality becomes 

Po i sup sup \R3{d,9)\ >n-A < e"™. 

\\d-d,\<w„ p-8<i.fc||<2ioc„ / 

Combining this result with (jB.8|) . it only remains to be shown that {d—do)gn,i{u) 
and [d — do)gn.2{u) are op^(l). Recall from CoroUarv 13.11 that \d — do\ = 
o(^^-(/9-i/2)/(2^i+i)) foj. 0. Consequently, 

\d-do\n^k-^ ^ o(n^+i/(2^+i)A:-5) ^ o((Vfce„)-^), {d-dofj = o((Vfce„)-i) 

k 

for all /3 > 2. This implies that Sn{d) = op^(l) and that, by Lemma |B.3[ 
{d — do)gn,i{u) = o(l). Also, for all (/i, I2) G {1, . . . , k}^ and all partitions a of 
(/i, I2), the limiting integral of tr [Ti^o-] is equal to 0. Since /3 > 5/2 the Lipschitz 
constants of the functions /d„,fc or fo are 0(1), so that Lemma 2.4 (supplement) 
implies tr [ri.o-((io, fc)] = 0{rfk). Similarly, 

n r 

tr [T2 ff(do, fc)] = 7;— / i?fc(a;) cos(Zia;) cos(/,a;)dx + 0(7i^fc). 
27r 

Thus we have 

k 



{d- do)gn,2{u) = ^h'^h I Hk{x)cos{lix)cos{ljx)dx + o{l) 



> ^ "ii^i^ / iifc( 

which is 0(1). This completes the proof of Lemma FB. II □ 
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The proof of the fohowing lemma is given in section 4 of the supplement. 
Lemma B.2. Let Wa-{d) denote any of the quadratic forms 

X'T-\fdM)Ba{d.ed,k)X - tr [Tn{fo)T-\fdM)BM,Sd,k)\ 
in (jB.2ll . For any j < J , {li, . . . ,lj) £ {0, . . . , fc}-' and a £ Sj, we have 

\WM)~W„{do)\^op^{\d^do\n^+'k-^), (B.12) 

tr[B„{d,ed,k)\ - tr[B,{doM] 

= {d- doMTiAdo, k)] + {d- do)2o(n'+U"H(i-^/2)+) (B.13) 
= (d - do)tr[Ti,„{do, k)] + {d- dofoin'-^/k), 
tr[{TMo)T-\fd,k) - I„)B,{d,9d,k)] - tr[{TMo)T-\fd^,k) - In)BMo,Od^,k)\ 
= (d - do)tr[T2.a{do, k)] + {d- dofo{n/k) + {d - do)o(n'+5fc-3). 

(B.14) 

Lemma B.3. For all [3 > 1 there exists a constant 5 > such that uniformly 
over \\0 - Od,k\\ < 5„, 

Proof. For u = 9 — 0d^k, we have 

9n,i{n) = -fr [T„(i/fc/,„,,-,^ JT-1(/,^^,-^^ jT„(u*V/,„,,-^^ jr-i(/,^^,-^^ J 

This follows from (jB.4p and Lemma below, by which b^r = d^^ = ^ and Co- = 
(the only partition for j = 1 being a = {{I})). By Lemma 2.4 (supplement) 
9n,i{u) converges to zero, but at a rate slower than n^^^~^k~^^^ . To obtain the 

o(n-'^/2-'5/j-i/2^ term, we write 

gnAu) = Ai + A2 + A3, 

and bound the terms on the right using the other lemmas in section 2 of the 
supplement. We first prove that 

Ai = tr [Tr,{Hkfd^M)Tnifl'k)Tn{u'Vfd^M)Tnifl]k) 

(16^4)tr [T„(i/fe)T„(w*cos)] = o(l), 
where cos(a;) = (1, cos(x), . . . , cos(fca;)). We then prove that 
A2 = tr[Tn{Hk)Tn{u* cos)] =0, 

and finally that 

A3 - tr [TniHkfd^,k)T-\fd„^k)TnWVfd^,k)T-\fd„^k)] 



~tr 



Tn{Hkfda,k)Tn 



1^ 



r„(u*v/d„,fe)r„ 



■fda.k 



o(ni/2-^fc-i/2). 
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To bound Ai we use Lemma 2.5 (supplement) with bi{x) = Hk{x), 62(2;) 
u*cos and L — Equation (2.6) then impUes that 

|Ai| < CVl||w||n'=(l + fc3/2-^fc-i/2) =0(1). 

To bound Delta2 note that for ^ = 0, . . . , fc and ah ji, j2 < n, 

n 

(T„(cOs(;x)))jij, = {Tn{Hk))j,,j^ = Vjh = \ji-j2\- 

j=k+l 

Therefore 



n n 



tT[TniHk)Tnicos{l.)]=J2Yl '^J ^i=bi -J2 |%i - j2 |=i 

ii=ii2=i j=fe+i 



since I < k and j > k. We now turn to A3. FoUowing iLieberman et al.l (|201lr ). 
we consider separately the positive and negative parts of Hj^ and of m'cos. Hence 
we may treat these functions as if they were positive. We first define, for fdk = 
(47r2/,„,fc)-i, 

A2 = T^{u'Vfd^,k)T-\fd.,k), B2 = T„(u*V/rf„,,)r„(/rf„,fc), 
A = Ti(iffe/d„,fc)T-i(/rf„,fc)ri(^^*V/d„,fc), 

B = TrUHkfd„^k)Tnifd„,k)T,Uu*Vfd„,k), 

A = /„ - T„(/d„,fc)T„(/d„,fc). 
Using the same computations as in ILieberman et al. ( 2011 ). we find that 

IA3I < |tr[BiB2A]| + |i-S||ri(i/fe/d„,fc)r„(/<j„,fc)Ari(w*V/d„,fc)| 
+ \A\^Vk\\u\\n' 
< y/k\\u\\n'k^^^-P + \tr[BiB2A]\. 

The first term on the right is o{n^^^^^k^^^'^). We bound the last term using 
Lemma 2.5 (supplement) with bi = _fffc, 62 ~ w*cos and 63 = 1, which implies 
that tr[BiB2A] = + 0{Vk\\u\\n'k^^/^~f^^+) = o(l). This achieves the proof of 
Lemma IB. 31 □ 



Lemma B.4. Suppose that k < fc„ and that l{fo, fdo,k) < ^o^n- Then all 
elements of\/iln{do, k) (I ^ 0, . . . ,k) are the sum of a centered quadratic form, 
S{yiln{do,k)) with a variance equal to ^(1 + o(l)) and a deterministic term, 
V{Vil„ido,k)) which is o(fc(3/2-/9)+„E). 

Proof. For all / = 0, . . . , fc, we have 

Vilnido, k) = S{Vlln{do, k)) + V{Vlln{do, k)), 
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where 

- fr [r„(/o)T-i(/<i„,fc)r„(Vi/d„,fe)r-i(/rf„,fc)] , 
v{ViUdo, k)) - itr [{TMo)T-\u,k) - /„) r„(Vz/rf„,fe)r„-i(/d„,fe)] . 

Note that this is a special case of (|B.2[) . with j = 1, fecr = c^o- = ^ and c^- = 0, 
the only partition being a = {{I}). The variance of SiV iln{do, k)) is equal to 

tr[(T„(/„)r-i(/d„,fe)r„(V,/<i„,fe)T-i(/rf„,fc))2] 

-(a;) ) cos2(Zx)dx + 0(n'fc), 



2^ 7-. V/do. 



since Lemma 2.4 (supplement) implies that the approximation error of the trace 
by its limiting integral is of order 0(n^(fc + fc^(^/^~'^'^'') = O(n'^fc). Since = 
g^da.k (^see (|A.ip ). the integral in the preceding equation is 



n 
2^ 



(1 + 2Ad^,k + 0(A^^ J) cos^{lx)dx 



— TT 



= ^ + 2naM) + 0{nSl)^^{l+o{l)), 

where ai is defined at the beginning of the supplement. Lemma 1.3 (supplement) 
then implies that the centered quadratic form is of order Op^(n'^+^/^). Similarly, 
Lemma 2.4 (supplement) implies that 

V{Viln{do,k))^^ r il^L^I^{x)cos{lx)dx + 0{n'\\Aa^,k\\ik) 



n 
2^ 



cos( 

— 7r 



i{lx)Ad„^k{x)dx + 0(nfc-2/3) + O^n'k^/^-'^) 
= 0(n^fc(3/2-/i)+) 

which completes the proof of Lemma IB. 41 □ 

Lemma B.5. Let A{d) he the (fc + 1) x (fc + 1) matrix with entries Ai^^i^{d) = 
ai-^+i^id), where ai{d) = \iyk(Qo.i^'2'l^^{do — d)). Suppose that k < kn and that 



KfoJd,k) < IWn. Then Md,k) = -V^Ud,k,e) 



satisfies 



< k, \{Jn{d,k)-Jn{do,k))i,.i^\ = op^{\d-do\n''k + Sn{d)) = opjn/fc) 

(B.15) 

uniformly over d £ (do — do + Vn) and k < kn- We also have for all /i, I2 
n n 

[Jn{do, k) - -h+i - ■7^A{do)\i^,u_ ■■= n{R2s)h,i2 + "(^2d)/i,i2, (B.16) 



31 



where (i?2s)ii,Z2 *s o, centered quadratic form of order o-p^in 1/2+"^) and (i?2d);i,'2 
is a deterministic term of order o{kn'^~^). For the matrix A, we have \\A{do)\\ = 
0(1) and \A{do)\ = 0(1). 

In particular, (jB.lGp implies that | J„(do, fc) — §/fc+i — §yl((io)| = op^(fcn^/^+'^) + 
o(fc^n'^). 

Proof. Let d and fc < fc„ be such that ^(/o, fd.k) !i ^o'^n so that d e (do ^ do + 
Vn) (see Corollarv l3.ip . Lemma FB . 1 1 implies that for all li,l2 < k, 

{Jn{d,k))i^^i^_ - {Jn{do,k))i^^i^ 

■.^-{d-do) Mr[Ti,,(do,fc)] + d,tr[T2,,(do,fc)]) + OpJ5„(d)). 

<Te5(Zi,/2) 

Lemma 2.4 (supplement) implies that 

tr[T,,,(do,fc)] = 0(fcn^), i = l,2 

so that (jB.lSP is satisfied since this term is op^ Xn^^^/k). We then use expression 
(|B.2p . with a e {({!}, {2}), ({1, 2})} and we denote cri and (T2 the first and 
the second partition respectively. Note that Ca^ = do-a = 1/2: Co-j = and 
do-i = 1- From Lemma 2.4 (supplement), the quadratic form in ( J„((io, fc));^^;^ 
is associated to a matrix whose Frobenius-norm is 0{y/n) and whose spectral 
norm is 0{n'^). Hence, this quadratic form is op^ (ri^/^+'^). Also by Lemma 2.4 
(supplement), the deterministic terms can be written as 

— / cos(Zia;) cos(Z2a;) (1 + Ad k) {x)dx + o{krf) 

ft 

= 2 (^h=i2 + O'h+h (do)) + o(fcn'), 
and Lemma IB. 5 1 is proved. □ 



C Proof of Lemma 3.5 

Under the conditions of Theorem 12.11 we have k = fc„ and /3 > 1, and we may 
assume (by Lemma [321) that l{fo,fd,k,e) < ^o^n- Fixing d and fc, we develop 
6 — ln{d, fc, 6) in 6d.k- From Lemma [B. II in Appendix [B] it follows that 

Md.M)-Md.^)=i: "'''"'"'r'-"'°-" 

7 = 1 

J - (C.l) 

+(d-do)5:^^^^^t^+5„(d), 
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where Sn{d) is as in (jB.5|) . Substituting (jC.l|) in the definition of In{d,k) in 
p.lj) . we obtain 



J\\0-0d.k\\<2loSr. 

||ll||<2ioi5n 

= 6''"'' ''n-e\k{0d„,k) / e^3=i 7T o; z.j=2 — ji t ^^y, 

J|t«|t<2/o<S„ 

(C.2) 

The first equality follows from the definition of /„ (d, k) and Leinma [3.3[ by which 
we may replace the domain of integration by {0 : ^d,fe|| < '^loSn}- The second 
equality follows from the assumptions on ng^k in prior A, the transformation 
u = 9 — 9d,k and substitution of (|C.1|) . Also the third equality follows from the 
assumptions on iTe\k' these imply that 

\log7rg\k{Od,k)-log7:eik{0d„M)\ = Mo-d||/i^7[fe] = 0(|rf-do|Kfc)5-^)+o(l), 

for some e > 0. Thus, the factor e^"'-'^\g^k{dd,k) on the second line of (jC.2| 
may be replaced by e'^"^''-'7rg|fc(0£i„.fc)- Because Sn{do) ~ op^(l), (|C.2p implies 
that 

J 



/„(do,fc) = (l + op„(l)) I expK.^ + V "^''^^'^."^^"'^^ \du. (C.3) 



||ti||<2;o'S„ 

The most involved part of the proof is to establish the bounds 



T^e\k[9da,k) / exp < hku + > V du 

J\\u\\<ioSr. I ~~i r- I 

^ T 1^1 i\ S„(d) (a \ f T'' , \u'-''>\jn„(d„,k)+(d-da)y-' ""■j'^' +fefe 



Mj|<2;oi5„ 



< T^e\k{9da.k) expihku+y } du. 



(C.4) 



Since the posterior distribution of 6 conditional on k ~ kn and d = do concen- 
trates at Od^^k at a rate bounded by /of^n (this follows from Lemma l3.2[ with the 
restriction to d = do), the left- and right-hand side of (jC.4p are asymptotically 
equal, up to a factor (1 + Op^(l)). By (|C.3p . the left- and right-hand side are 
actually equal to In{do,k). This implies that /„((i, fc) = e'^" (do, fc), which 
is the required result. 
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In the remainder we prove (jC.4[) . To do so we construct below a change of 
variables v = ip{u), which satisfies 



'^^^ + 2^ -\ = /ifcu + 2^ 



.7 



(C.5) 



.=2 ^ 



for all < 2loSn. We first define the notation required in the definition of ip in 
(jC.Sp below. Recall from (|B.4p in Lemma [B . II that gn.j{u) can be decomposed 
as 



k 



(T^Sj li,...,lj—0 

where g^^^^ j depends on a. For ease of presentation however we omit this de- 



pendence in the notation. Using Lemma 2.4 (supplement) and (|B.4p in Lemma 
[BJl it follows that for aU j > 2 and {h, . . . e {0, . . . , ky , 



7;'^^^ ij ^ 2~ J Hk{x) cos{lix) ■ ■ ■ cos{ljx)dx. 
Let G^^^ denote the matrix with elements 7;^''; , and G^^^ the matrix with ele- 

(2) 

ments g^_^ \^ . By direct calculation it follows that 

Similarly, for all j > 3 and li, . . . ,lj G {0, 1, . . . , fc} we define 

fc k 
/3,...,/j=0 la,---,lj=a 

In contrast to G^^) and G^^), G(j)(u) and G^^H") depend on u. For notational 
convenience we will also write G'^^''(u) and G^^-'(u). Finally, let Ik = Jn{do, k)/n 
be the normalized Fisher information. 
We now define the transformation ■0: 

V'(u) = (/fe+l - (d - do)i:'(M))M, with (C.8) 

j=2 ^ ■ o-G5j 
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= — r— yry ^ Ui^ . . . Ui^V i,^„jJn{do, k). (C.9) 
j=3 '^^^ h,-dj=0 

The construction of G{u) is such that 

,/ 

nu*G{u)u^"^g„j{u). (C.IO) 

Analogous to G(it) and D{u) we define 0(1*) = X]/=2 jl X](tg5 ('^'^ ^ da)G'^^\u) 
and -D(u) = (7^ + L{u))^^&{u). After substitution of = ^^nd using 

(|C.25p in Lemma FC. II it follows that 

^ {v(3) -u^j))vn„{d,,k) _ 

The definitions of -D(m) and L{u) and (jC.lOP imply that 

- n{v - ufiku = n{d - do)u* D*{u)ikU = n{d - do)u*G{u){ik + L{u))~^IkU 
= n{d - d^yGiu) (4+1 - {Ik + L(u))-iL(ii)) u 

1 

= {d- do) E —9n.j{u) - n{d - do){D[u)uf L{u)u 



J 

r- 

'^h,...,lj ln{do,k) 



J=2 

,7 J-1 fe 



{d- do)^^gn,3{u) - {d~ do)Y^ ^ 



At the same time, the definition of Ik implies that 
Combining the preceding results, we find that 

^^■^ + 2^ \hku + }^ + (rf - do) ^ —j^ 

= hk{v -U) + {V- ufVUdo, k) - <^-^yi^k{v-u) ^ 

= («-u)*v;„K,fc) + o(5„(d)), 

where the last equality follows from (|C.24p below in Lemma [C31 together with 
the assumption on in prior A in 
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Apart from the term [v ~ uY'Vlnido, k) on the last line, the preceding display 
implies (jC.5P . Hence, to complete the proof of (|C.5p it suffices to show that 

[v - ufVUdo, k) = -{d - do)u'D'iu)VUdo, k) = 0(^„(d)). (C.ll) 

The proof of (|C.11[) consists of the following steps: 

\u\D{u)-D{u)fVln{do,k)\ = opJn^-*fc-^), (C.12) 
{d-do)v'D\u)V{mn{do.k)) = 0{Sn{d)), (C.13) 
{d~do)u'D\u)S{mn{do,k)) = 0(5„(d)), (C.14) 

where S {yin{do,k)) denotes the centered quadratic form in V/„((io,fc), and 
'D (VZ„(do, k)) the remaining deterministic term. We will use the same notation 
below for L{u). 

Equation (|C.12p follows from Lemma IB. 41 and (|C.22p in Lemma IC.ll be- 
low, which imply that the left-hand side equals o-p^{{y/k\\u\[f'rr'^^^k^/n) = 
op^(n3~''fc~2 )^ for some (5 > 0. For the proof of (jC.lSp . note that Lemma [8.41 
implies 

\u'D\u)V{Vln{do,k))\ < ||^(u)u||\/fcfc(3/2-/3)++^. 

Combined with Lemma fCH this implies that the left-hand side is 0{Vkk^^^~'^^~^'^), 
which is 0{Sn{d)). The proof of (|C.14p is more involved. Recall that D{u) is 
defined as D{u) = {Ik + L(u))~^&{u). Using (jB.16p in Lemma [B.Si we obtain 

(7fc + L{u))-^ = 2[/fe+i - {A{do) + R2s + R2d + L{u)){l + op„(l))]. 

Substituting this in D{u), it follows that (jC.14p can be proved by controlling 
G«5(VZ„(do)), G(^)A(d„)5(V/„(do)), G(^)i?2d5(V/„K)), G(^')l?(iH)5(V?„(d„)), 
G(^)i?2s5(VZ„(do)) and G'^^^S{L{u))S{yin{do)) for all j = 3, . . . , J. To do so, 
first note that Lemma lB.51 implies that ||G^-'-'i?2s'5(VZ„((io))|| = op^{n'^Vk). 
Hence 

|u*G(^')i?2s5(V/„(do))| = op„(fc-^+i+') = Op„(l), 
which clearly is 0(5„(d)). The terms G(j)5(V/„(do)), G^^^ A{do)S{Vln{do)), 
G'^^^ R2dS{yin{do)) and G^^^'D{L{u))S{Vln{do)) can be written as quadratic 
forms Z*MZ — tr[M], where, for a sequence {bi)f^Q and a function g with 
llolloo < oo, M is of the form 



i cos{lx)j T-\fa^^k)Tr?{fo), 

Z being a vector of n independent standard Gaussian random variables. Using 
Lemma 2.4 (supplement) it can be seen that |M|^ < ni^^b^ + k/n). Lemma 
1.3 (supplement) with a = e + 1/2 then implies that 

Po {^Z'MZ -tr[M]\ >n^+5 (^bi + ^ j < e"™'. (C.15) 

For all j G {3, . . . , J}, the four terms above can now be bounded for a particular 
choice of g and bi . 



36 



1 aii i^, . . . , c x^, . . . , n,j, act — 

and g{x) = 1. Then we have 



BoundonG(j)5(VZ„(do)). For all/a, S {0, . . . , fc}, set = t}^^,;,,^....,,. 



1=0 

By induction it can be shown that 
1 

2^ 



cos(Zo2;) cos(/ix) cos(Z2a;) ■ ■ ■ cos{ljx)dx 

ei,...,eje{-14} 



(C.16) 



Consequently, J^i^f = '^(^ ^) fo'" hih, ■ ■ ■ ,lj € {0,...,fc}. Us- 
ing the fact that J^i l^il = o(l), we obtain that (G(^'5(V/„(do)))(, = 
0Po(ni/2+e^-i/2-)^ foj. j^ii /2 € {0, . . . , fc}. This imphes that 

\\&^^S{VUdom = op„(ni/2+^) = 0{S„{d)). (C.17) 

Bound on A{do)S{Vln{do)). Set 6; = I(+(i>A;^'o,i+ii and g{x) = 1, then 

{A{do)S{Wln{do)))i, = op^{n^/^+'k-^), Vh e {0, . . . , k}. 
Combined with Lemma IC.ll this imphes that 

||G(^'A(d„)5(V/„(do))|| = op„(ni/2+^A;-^+i/2). 

Bound on R2dS{\7ln{do). Set 6; = {R2d)h,i and .g(x) = 1, for all 
li = 0, . . . , fc, then Lemmas IB . 5 1 and IC . 1 1 lead to 

\\&^^R2dS{yin{do))\\ = op„(ni/2+^fc2n-i) - op„(n-i/2+.fc2)^ 

Bound on G'^^'^V{L{u))S{Vln{do)). For all /i, /g, . . . , Zj, /g, . . . , e {0, . . . , fc} 
set 



6; = -tr 
n 



\i2=0 / 

T-\fdjT„{cosilx)g2ix)fdAx)) ■ ■ ■ T-\fd^)T„igr{x)UAx))] 



where gi{x)^ gr{x) are products of functions of the form cos(/ja;) and 
gi(x)....gr{x) = cos{l^x)...cos{l_.,x). Lemmas 2.1 and 2.6 in the supple- 
ment, together with (jC.16[) . imply that 

J2^f = and \\G^^'>V{L{u))S{Vlnidom = Op^{n'+^^^). 

(C.18) 
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Consequently, the contribution to all these terms in {v — uYVlnido, k) is of 
order 0(5„(d)). 

We control u*G(j')5(i(ii))5(VZ„(do, fc))_, by bounding ||G(^'5(L(m))|| using 
a similar idea. Indeed, for all li,l2 < k, {G'-^''S{L{u)))i-^j^ can be written as a 
sum of terms of the form {Z*Mi-^j^Z — tr{Mi-^^i^))/n, where Z is a vector of n 
independent standard Gaussian random variables, and Mi-^^i^ has the form 

Thfo)T-\fd^,k) (]lT,,{W,{t)U^.k)T-\f,^,k)] X 

\i<io / 
\l=0 / i<io 

We can use the same argument as in (|C.15p since for all I1J2, ■ ■ ■ Jj-i 

k 

< \Tnhh.,kTn{Y.jl^ili,_i^_, COs(/x)V,(,„)_„/rf„,fc)r« 



1=0 



Hence, it follows that n-^[Z* Mi^^i^Z - tr{Mi^,i^)] = op^(n~i/2+e;(,-i/2) g^^^^ 
v'GS{L{u))S{Vln{do,k)) = op^iWuWn'k) = op^{n^'^-^k-^'^). (C.19) 

Combining ((CJ9)) and ((CTf|) - (|C?T8| . we obtain ((C?T4| . This in turn finishes 
the proof of ()C.lip . since 

{v - ufS/ln{do, k) = op„(|d - do\n^-^k-'^) = 0{Sn{d)). 

We now prove that ^{u) is a one-to-one transformation. First note that 
ij}{u) is continuously difFerentiable for all ||m|| < 2Zo(5„. This follows from the 
definition ■0(it) = {Ik+i — {d— do){Ik + L{u))^^G^{u))u, the fact that G{u) and 
L{u) are polynomial in u and Lemma I C. 11 by which |jL(M)|| = op^(l). To prove 
that ■)/'('") is also one-to-one, we bound the spectral norm of the Jacobian 

^'(u) = h+i -id- do)D{u) -{d- do){D'{u)u), 

where {D'(u)u) is the (fc + 1) x (fc + 1) matrix with elements 

' d{D{u))i,,i 
2_^ui , li,l2 = 0, ■ ■ ■ ,k. 

;=o "^^^ 

For ipiu) to be one-to-one, it suffices to have ip'{u) — /fc+i(l + op^(l)). 

By ((024)) in Lemma [CH below, we have \d - do|||i:»(u)|| = Op^{\d - do\). 
Therefore we only need to control the spectral norm of D'{u)u. For all li, I2, we 
have 



-{Ik + mr'^^ih + L{u))-^G\u)u + Ch + L{u))-'^^p^, 

(C.20) 
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Both iG{u))i-^^i^ and {L{u))i-^^i^ can be written as 

,7 k 

where the constants Tj, bi-^^,,,^i. are different for G and L, and b is symmetric in 
its indices. In particular, T2 ~ in the case of L. Using this generic notation 
for G{u) and L{u), we find that for aU v £ R'^+^ and ah li,l2 < k, 

dF{u;T,b)v\ ^- o , IN z, / M 

= Z^'^jU^3 + l) 2^ vi^ui_^- ■ ■ui.hi^^i^^....i^ := F[v,u;t ,b), 



duu 



j=3 i3,...,ij=0 



where t^- = Tj(j — 3 + 1), j = 3, . . . , J. It therefore has the same form as 
F{u;T',b), with v replacing one of the u's. Applying this to the first term of 
([Ul20)) . with V = {Ik + L{u)y^G*{u)u, we find that 

\{Ik + L{u))-^F{v,u;T\b)\ < \F{v,u;T\b)\ =0{1), 

where we used (|C.2ip and ()C.24p from Lemma ICH The second term of (|C.20p 
is treated similarly with w = u so that we finally obtain 

\D'{u)u\ = 0(1), 

and ip is one-to-one on {u : \\u\\ < 2lQSn}- Using the above bounds we also 
deduce that the Jacobian is equal to exp{0{Sn{d))), since 

logdet[Jac] = logdet [Ik+i — {d — do)D{u) — (c? — do)D' {u)u\ 

= 0[{d - do){MD{u)]\ + tT[D'{u)u]) + {d- dof{\D{u)\^ + \D'{u)u\^))] 

= 0{Vk{d - do) + k{d - dof) = 0{SrM))- 

This finishes the proof of \CA\ , and hence the proof of Lemma 13.51 

Lemma C.l. Let v ~ ipi^)y with as in (jC.Sp . Under the conditions of Lemma 
Iff. 51 we have 

\L{u)\ = op„(n-i/2+^fc) =opJl), (C.21) 

\G-G\ = op„(j7-i/2+^fc) -opjl), (C.22) 

\D{u)\ = Op„(l), (C.23) 

h-Hu)\\ < \d-do\OpM\), (C.24) 



(t,(j)-^(j))VJ^„(d„,fc) 

j=3.h,...dj=0 >' 



(C.25) 



uniformly over \\u\\ < 2loS„ 
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Proof. We first prove (|C.2ip . From (|B.2[) . we recall that Vi^^,,,j.ln{do,k) is the 
sum of a centered quadratic form S{V i-^^,,,^iJnido^ k)) and a deterministic term 
T^[^h,...,uln{do,k)). For all S(yi^^,„jJn{do,k)) equals 



with Ba{do) '■= Ba{do, as defined in (|B.1I) . Using Lemma 1.3 (supplement) 

together with (|B.10p we obtain that for all 5(V;j...._; J„((io, fc)) = 

Op^(n5+'), and its contribution to is Op^{k{\/k\\u\\y^'^n^^^^) = Op_^(n^^/^^ 

The deterministic term in (jB.2p is 

V(yi,,...,iMdo,k)) = ^c,tr [B,{do)] +Y,datr [{Tn{fo)T~\fd^,k) - 



We bound the contribution of the first term to |L(u)|; the second term can be 
treated similarly. Let L{u) be the matrix when in (jC.9|) we replace Vij^...^;^J„(do, k) 
by E<TC<Ttr[B^(do)]. Hence, 

iL(u)\,. = -t7-hv^'^ ^ (C.26) 

3=3 ' <y£Sj l3,...,lj=0 

1 1 r 

where —tr [5^(^0)1 = — / cos(Zix) . . . cos(/,a;)(ia- + E'er, (C.27) 
" 27r 

Ea being the approximation error. For each a and j > 4. the contribution of 
the integral in (|C.27P to {L{u))i,j^ is 0{J^^ lu'cosp'^^^^.^^^,) ^ 0{\\u\\^); hence 

its contribution to is = o{n~^^'^~^k). For j = 3, we have 

^ X! / cos(;ia;) cos(Z2a;) cos(;3a;)da; = - (m^+i^li^^i^+i^ + uii^_i^i\^ii^_i^i) 

l3 = l 



and the contribution of this term to is of order \/fc||u|| = o(n^^/^+^fc). 

Next we bound the contribution to |i(w)| of the error term Ea in (jC.27p . Note 
that we can write the last sum in (jC.26P as 



^ tr[B,(do)] 1 
> ui^...ui^ = -tr 

h,...,lj=0 



nT„(6,(x)/<i„,fe)T-i(/,„,fe) 



where 



h{x) = (u*cos(x))l''WI-''i(')^''^(') cos(;r)''^^'^ cos(Z2-)^'^'\ 



, (C.28) 



(C.29) 



Si{i) = I[ie<T(j)' '^2(*) = ^2ea(i) and p = |(t|. If p < 3, then Lemma 2.4 (supple- 
ment) implies that 



E, = 0{{Vk\\u\\y-^n'-^[k^^^/^~^^+ + k]) = o{n-^~^k). 



(C.30) 
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If p > 4, then Lemma 2.6 (supplement) together with (|C.28[) . with 

L = A:(3/2-/3)+^ Af,TO^i = 0(1), A/W = 0{{^/k\\u\\)\'''''^\'^^^'^'^~^^'^''^ and 
0(A:(\/fc||w||)l'"WI-'^iW-''^W), leads to the bound 



A tr[S,(do)] 1 
"'3 ; -tr 



- 0(fc(3/4-^i/2)+„-l/2+.||^||(^||^||)J-3) ^ o(„-l/2-5)_ 

Using Lemma 2.1 (supplement) we finally obtain that 



1 



-tr 



1=1 



1 

2^ 



(u cos)-' (x) cos{lix) cos{l2x)dx ~ o{n k) 



Therefore the contribution of the approximation error in |i(w)| is of or- 
der o{n~^^'^~^k). Using a similar argument we control the terms in the form 
tr [Tnifo)iT-\do,k)_In)B,{do)] and (ICl2T]l is proved. 
We now prove ((a22| and bound 



U) 



j=2 creSj 



l3,...,li=0 



with r|^' J as in (jC.6|) . These are the approximation errors which occur when 
replacing ■^ti[Ti ,y{do, k)] and ^ti-[T2^a{do, k)] by their limiting integrals (see also 



4|). Therefore, for each a G 5j, J^i-^ 1=0 



So) 



of terms of the form 

p 



ii,...,;,"'3 • 



.ui, is a combination 



1 

-tr 

n 



[]r„(5,(a:)/rf„.fc)T-i(/d„,fe) 



.1=1 



1 

/ {u^cos{x)y^'^Hk[x)cos{lix)cos{l2x)dx, 

27r 



with pS {|cr|,|cr|-|-l} and the functions bi defined as in (|C.29[) apart from 
61 (x) = i7fc(x)(^f^ou,cos(/x))l'^(i)l-*i(i)-*^(i)cos(/i.)*i(i)cos(/2.)*^(i). There- 
fore, using the same construction as in (|C.28P - (jC.30p . we obtain that 



|(G - G)ia. I = 0{n-^'^+^), \G-G\= 0{r 



fc)=o(l). 



To prove (jC.23p . we use the just obtained bound on jC — Gj, and in addition 
establish a bound \G\. We treat each term G^^-* in G{u) ~ X]j=2 yr X^ues ("^o- ^ 
da)G'^^^{%i) separately. First we show that \&^^\ = 0(1), which follows from 
definition ()C.7p . by which 



tr 



(G(2))2 



E 



'l,'2:'l+'2>fc 



. (^1+^2)^ 



< 1. 
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Consequently, \G^'^^ \ < 1. For j > 3, note that for all Q<li,h<k, 

k 



k 



< 



^ \uu...ui^\ r \Hk{x)\ 



li....,li=a 



cos{l3x)ui. 



h=0 



dx 



Therefore, \G^^\u)\ < k{Vky^'^(\\u\\y-^ = {Vk\\u\\y-^ = o(l), for all j > 
3. Hence \G{u)\ = 0(1), which combined with || (/+L(ii))-l|| = Op^(l) (see 
Lemma rB.5[) and (jC.21[) ). imply that 

\Diu)\ = \ih+Liu))-'&iu)\=Oil), 

uniformly over ||it|| < 21qS„. It follows that 

\D{u)\ < 11(4 + L{u))-'\\ (|G| + \G-G\) = Op„(l). 

This concludes the proof of (|C.23p ; (jC.24[) directly follows from this result since 
\\u-'ip{u)\\ < \d-do\\D{u)\\\u\\. Finally, we prove jC^li. We have 

^ (^b-)_^b))VJ7„(d„,fc) ^ ' V,„...,,J„(do,fc) 

2^ n = -id-do)2_^ 2^ {D{u)u)i,ui,...ui^- 

j=3 j=3 ;i:...,/j=o 



{d-d,fY.[ ] E {D{u)uyAD{u)u) 



'^h,....l, ln{do,k) 



3=3 ^ ' h,-dj=0 
k 



(j - 1)! 

....,ljln{i 



+ ... + {-iy ^ {D{u)uy, {D{u)uy^ . . ■ {D{u)uy, ^^yM^;" 

li,...,lj=0 

Using the same argument as in the proof of (jC.2ip . we find that for all for all 
J >3 

fe 



J2 iD{u)uy,{D{ 



u)u)i^ui^ ...Ul, 



^h,...,ldn{do, k) 



/i,...,;,=o 



= n {D{u)uYcos{x)y{u*cos{x)y-^dx 

J —TT 

+ {y/k\\u\\y-^0{V^n'{Vk\\u\\) + k + VH||w|lfci/2(3/2-/J)+) 
= o{n^~^k~^), for some S > 0. 

Similarly, the higher-order terms in the above expression for J2j=3 ~ — ~ — )v i„(do,fc) 
can be shown to be 0{Sn{d)), which terminates the proof of Lemma FC. II □ 

The rest of the paper corresponds to the suppelmentary material 
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D Technical results 

Let rjj = — lj>o2/7 and recall that 6d,k = ^o[fc] + {do — c^)'?[fc]- Let the sequence 
{aj} be defined as Uj = Oo.j + {do — d)rij when j > k and aj = when j < k. 
In addition, define 

oo k 

Hk{x) = ^ 7]^cos{jx), G'fc(a;) ^77jCos(jx), (D.l) 

J=fc+1 3=1 

oo CO 

^d,k{x) = ^ (6*0 J + (do - d)77j)cos(j.T) = ^ a-, cos(jx). (D.2) 

j=k+l j=k+l 

Using this notation we can write 

-21og|l-e"| =-log(2-2cos(x)) -Gfc(a-)+iffc(x), (D.3) 



(D.4) 



fd,k{x) = fd,k,e^Ax) = fo{x) exp <^ - ^ aj cos(jx) 

Given d,k and Oo, the sequence {aj} represents the closest possible distance 
between fo and fd.k.e, since 

KfoJd.k) = KfoJd^kj.J = ^ ^lk{^)dx = 



From (ID. 41) it also follows that for all d, 



d 

^/ti,fc = Hkfd,k- (D-6) 

Lemma D.l. When Oq G Q{f3,Lo), there exist constants such that for any 
positive integer k, 

k'^ < f Hl{x)dx < k'\ (D.l) 

J2\eod\ - 0(fc-^+^), ^|0,,|=O(l), (D.8) 

l>k l>0 

Ad^^k{x)Hk{x)dx = J2r^^9o,,^o(k-^j , (D.9) 

j>k 

r Al^,ix)dx = J2<i-0{k~'^), (D.IO) 
•'-^ i>k 

Al^,{x)Hk{x)dx = 0(^-2/5-1), (D.ll) 
H^(x)dx < i^^. (D.12) 
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When k ^ oo, the big-0 in ([D^ - (|DlT|l may be replaced by a small-o, since 
^i>k i^"^^ then tends to zero. 

Proof. The result for J H^{x)dx follows directly from the definition of Hk. The 
assumption that 60 G 0(/3,io) and the Cauchy-Schwarz inequality imply that 



i>k y i>k y ;>fc 

proving the first result in (jP.SP . Similarly, one can prove (fPlQ]) . For (jDlQ]) . note 
that X)i>fc / — fc"^*^ E;>fc i^^*^- For the other bounds we omit the details 
of the proof. They follow from the fact that for all sequences a, b and c, 

2 E] o-ibmCn j cos{lx) coa{mx) cos{nx)dx 

— E] ^mC„ a/ / cos(Za:;) (cos((m + + cos((m — 

m,n>k l>k '' 

m,n>k m,n>/i;;m — ?i>fc 

□ 

Before stating the next lemma we give bounds for the functions Hk and Gk- 
Since — 21og |1 — e^^\ = — log(x^ + 0(x'^)), there exist positive constants c, Bq, 
Bi and such that 

\Hkix)\ > Bo\ \ogx\, \x\ < ck-\ (D.13) 

\Hk{x)\<Bi\\ogx\+B2\ogk, xG[-7r,^]. (D.14) 

Lemma D.2. Let aj = {60. j — (d — do)rij)lj>k, as in (jD.2|) . T/ien /or p > 1 
and g = 2,3,4 there exist constants c{p,q) such that for all d G (^^i^) and 
k < exp(|d — do\~^), 



fdA^)J V ^ / (D.15) 

+ O((logfc)«+f^^l'*-'^°l|d-dor5e-l'i-'^°l"'), 

27r / ijdk^^^ ~ cos(ja:)dx = ^ai+jh+j>k + O |^E j ' 

(D.16) 

^ f {jfl^'')-^Hki^)'^^^0{\d-do\k-Hogk), (D.17) 

where the constant B2 in (jP.lSP is as in (jD.14p . and i/ie constants in (jD.16[) 
and (jP.lTP are uniform in d. The constant c{p, q) in (|D.15P equals 0, ^, 1 when 
respectively q ~ 2,3, 4. 
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Proof. When d = do, (|D.15|) directly follows from (|D.7|) and (|D.12p . because 
of the boundedness of {fo/ fd„,kY = exp{pAd^_fc}. Now suppose d ^ do- Let 
Ck = max^g[_^ cxp{|Ad^,fc(a;)|} and = max^-gj^^^j |(d - do)iJfe(a;)|, for 

jYi = \d-da\ < e^^. Since X^jlp l^ojl < co, the sequence Ck is bounded by 
some constant C . To prove IjD.lSp we write 

...... X . (D.18) 



fd,kix)J 7„ \fdAx) 

We first bound the last integral in the preceding display, by substitution of 
{fo/fd,kY = exp{pAd,fe} = exp{-p(d-do)i7fc+pAd„,fc}. From (IdHI it follows 
that 

6,n < \d-do\{B^\d-do\-^ ^B2\Qgk) < B1+B2, 

k < exp{\d— do\~^). Hence we obtain {Jo/ fd,kY < Ce''" on (tojTt). For q~2 
and 9 = 4 the bound on the last integral in (jP.lSP therefore follows from ()D.7P 
and (|D.12[) : for g = 3 the bound follows from the Cauchy-Schwarz inequality. 

Next we bound the first integral in (jP.lSp . Because the function x'''"''"! (log a;)^ 
has a local maximum oi4\d-do\~'^e~'^ at x = e-^/M-dol ^ < ^^-\d-d^\ |^_ 

do|~^e~^ for all x £ [0,m]. Again using (|D.14p we find that 



ilM))' 1^^-''^'^)'^'^ - i C) (^il log^l)' logfc)'-^ e 
^ I] (j) (logA:)«-J+P^^I'^-'*°l (Bil logxl)^' x-f^^l'^-'^°ldx 



p-p{d-da)Hk(x 



J=0 

< (log/c)9+P^=^l''-'^°l|d-dor5e-i/l^-'^°l. 
We now prove (jD.16p . 

1 r / /o 



ix) — 1 cos(ix) cos(7x)(ix 



= ^ j {e^'^'''^^^ — cos{ix)cos{jx)dx 

< i- y" (^Ad,fc(x) + ^A^ fc(x)e(^^ '=("»+^ cos(ix) cos{jx)dx. 
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The linear term equals 

1 r 

— / Ad^k{x)cos{ix)coa{jx)dx 



T~ / iy^.^-i cos{lx) I (cos((i + j)x) + cos((i - j)x)) dx = \ai+j U+j^k ■ 
4^ ^-^ \tt J 2 



For the quadratic term we have 

j,(a;)e('^''''=('^^)+ cos(ia;) cos{jx)dx 



1 

2^ 



27r 



fc(x)da;. 

(D.19) 



This is 0(J2j>k '^^)^ which follows from (jP.SP and integration over (0, e ) 
and (e~'^,7r) as above. 

To prove (iDTTl) . write exp(Arf,fc) - 1 = Ad,k + Ajj^ e^ with A^^fc = -(^ - 
do)Hk{x) + Ac;^^fc(a;) and jd — do\ < w„, substitute (jD.14p and proceed as in the 
proof of (jD.15[) above. The biggest term is a multiple of jd — doj J^^ \Hk{x)\^dx, 
which is 0{vnk^^)- This is larger than the approximation error when /? > 

i(l + \/2). □ 



Lemma D.3. Let A be a symmetric matrix matrix such that |A| = 1 and let 
Y = (Yi, . . . , Y„) be a vector of independent standard normal random variables. 
Then for any a > 0, 

P {Y'AY - tr{A) > n") < exp{-n"/8}. 



Proof Note that \\A\\ < |yl| = 1 so that for all s < 1/4, sy^Ay < soy*y\\A\\ < 
and exp{sF*Ay} has finite expectation. Choose s = 1/4, then by 
Markov's inequality. 

P {Y*AY - tr(A) > n") < 



< 

The last inequality follows from the fact that A{In — tA/2)~-^ has eigenvalues 
Aj(l — where Xj are the eigenvalues of A for all t G (0, 1). Hence, 

tr(A^(/„ - tA/2)~^) is bounded by 4tr(A^). The result follows from the fact 
that when n is large enough > 2tr(A^) =2. □ 



exp |-n"/4 - i logdet[/„ - A/2] - tr(v4)/4 
exp{-n"/4 + tr(A2)/4} . 
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E Convergence of the trace of a product of Toeplitz 
matrices 



Suppose Tn{fj) {j — 1, . . . ,p) are covariance matrices associated with spectral 
densit ies fj. According to a classical result by Grenander and Szego ( Grenander and Szegol 



-tr 



{2tt 



>2p-l 



In this section we giv e a series of related results. We first recall a result from 
Rousseau et al. I (I2OIOI) . 



Lemma E.l. Let 1/2 > i > and LW,MW > 0, p, € (0,1], d, € [-1/2 + 
t, 1/2— t] for alii — 1, 2p and let fi, (i <2p) be Junctions on [— tt, tt] satisfying 

= W^'^g^ix)^ \g,{x)\ < \g4x)-g4y)\ < 

(E.l) 

and assume that X]r=i(^2i-i + c^2j) < 5- Then for all e > there exists a 
constant K depending only on e,t and q — X]j=i('^2i-i ^2j)+ such that 



1 

-tr 

n 



i[TM2,-l)Tnif2j) 



2p 



(27r 



TT 2p 



2p 



i=2 V'5^J 



1=1 



To prove a similar result involving also inverses of matrices, we need the fol- 
lowing two lemmas. They can be found elsewhere, but as we make frequent use 
of them they are included for easy reference and are formulated in a way better 



suited to our purpose. The first lemma can be found on p.l9 of lRousseau et al 
and is an extension of Lemma 5.2 in iDahlhausI (Il989l) . 

Lemma E.2. Suppose that for < t < 1/2 and d S [-1/2 + t,l/2~t] 

\f{x)\^\xr^''g{x), m<\g{x)\<M, \g{x) - g{y)\ < L\x - y\P (E.2) 

and assume that < m < 1 < Al < +00 and L > 1. Then, for all e > 0, there 
exists a constant K depending on t and e only such that 



\In-T^{f)Tn 



1 



Any 



1 9 , 



Proof. By Lemma IE. 11 
/ 1 



+Ti(/)r„ 



V47r2/ 
1 

4^ 



Tn{f)Tn 



tr<! /, 
1 



4^2/ 



,-2T„5(/)T„ 
THf) 



4^2/ 



T^if) 
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converges to zero, the approximation error being bounded by K[L{1+ M'^ /■m'^) + 



The next resuh can be foun d as Lem ma 3 in iLieberman et al.l (|201lh . and 
is an extension of Lemma 5.3 in iDahlhau s (198!|). 



Lemma E.3. Suppose that fi and /2 are such that > mlxl ^''^ and 

|/2(x)| < M\x\~^'^'' for constants di,d2 G (-i, ^) and m,M > 0. Then 

TO 

Proof. In the proof of Lemma 5.3 on p. 1761 in Dahlhaus ( 1989f ). the first 
inequahty only depends on the upper and fower bounds to and Al. □ 



Using the preceding lemmas, the approximation result given in Lemma IE. II for 
traces of matrix products can be extended to include matrix inverses. 

Lemma E.4. Suppose that f satisfies (jE.2p with constants d, p, L, in and M . 
For f2j, j = 1, . ■ . ,p, assume that (jE.ip holds with constants d2j, P2j, L^^^^ and 
7l^(2i) — 1, . . . For convenience, we denote M^^-'"^-' = m^^ , P2j-i = P 
and L^'^^~^^ — L (j ^ 1, . . . ,p). Suppose in addition that d, d2j G [—5 + t,j—t] 
satisfy X]j=i('^2j - d)+ < i(p - i), and let q I]j=i(^2j - d)+. Then for all 
e > there exists a constant K such that 



i..{n--(/)T„(/.,}-^£n 



f2j(x) 
fix) 



dx 



< K 



2p / 2p 

i<2p 

(p+i) 




(E.3) 



,(i-p) 



(p+i) 



■l+c+2(2 



and setting f ~ l/(47r^/), 



tr^ 



J[TM)Tn{f,) 




(l-p)i£±ii-l+e+2g 



(E.4) 



Proof. Without loss of generality, we consider the f2j 's to be nonnegative When 
this is not the case, we write f2-i = fy — f oj and treat the positive and negative 



part separately; see also IDahlhaus! (I1989D . p. 1755-56. To prove (|E.4[1 . we use 
the construction of Lemma 5 from iLieberman et al.l (|201l[ ). who treat the case 
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p = 1 and d2j = d' . Inspection of their proof shows that this extends to p ^ 1 
and d2j that differ with j. To prove (|E.3p . we u se the construction of Dahlhaus ' 
Theorem 5.1 (see also the remark on p. 744 of iLieberman and Phillipj ( 20041 ). 
after (28)), and apply Lemma IkD with /2j-i = / = j — 1, • • ■ This 

gives the first term on the right in (jE.3p . The last term in (jE.3[) follows from 
(El. □ 



Although the bound provided by Lemma IE.4I is sufficiently tight for most 
purposes, certain applications require sharper bounds. These can only be ob- 
tained if we exploit specific properties of / and /2j. In Lemma IE. 51 below we 
improve on the first term on the right in (|E.3|) . This is useful when for example 
hi{x) ~ cos{jx); the Lipschitz constant L is then of order 0{k), but the bound- 
edness of bi actually allows a better result. In Lemma lE. 61 we improve on the 
last term of (|E.3|) . 

Lemma E.5. Let f{x) = \x\-~'^'^g[x) with -1/2 < d < 1/2 and g a bounded 
Lipschitz function satisfying m < g < M , with Lipschitz constant L. 



• Let hi, . . . ,bp be bounded functions and let \\b\\ 
bound for these functions. Then for all e > 0, 



denote a common upper 



tr 



p 



Y[T„ihf)TM-') 

.i=l 



{2TTftr 



\{Tn{h) 



:i=l 



< Cn" 



h\W I \\h\\oo + LY^\\b,\\2 



(E.5) 



• Let bj {j > 2) be bounded functions. Let bi be such that \\bi\\2 < +oo, and 
assume that for all a > there exists M' [a) > such that 



\bi{x)\\x\~^+''dx < M'{a). 



Then for all a > 
p 



tr 



'[[tm)tm-') 



{2Tr)Ptr 



t[Tn{bi 



.i=l 



< c 



M 



ll\M^{n^P^M'{a)+L{\ognfP-^b,h) 



(E.I 



Proof. We prove (|E.5P : the proof of (jE.6p follows exactly the same lines. We 
define A„(a;) = e'^ and L„(x) = n A \x\^^ where the la tter is an upper bound 
of the former. Using the decomposition as on p. 1761 in iDahlhaus (|l989f ) or as 
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in the proof of we find that 

p 



tr 



< C 



(27r)Ptr 



Y[TM)Tnif-' 

i=l 

r ^ I ^ 

I nMx--i) n 



f{x2i- 



\ 1 ) A„(xi ~ X2) ■■■ A„(a;2p - Xi)d2 



< C 



C 



^=l 



\X2i-l\ 



/ n^^(---i) n 



IT- 50f2i-l) 



\X2i 



\-2d 



1 An{xi - X2) ■■■ A„(x2p - xi)dx 



1 A„(xi - 2:2) . . . A„(x2p - xi)dx 



< C 



fM\\b\\ 



CL 



\ m 

M\\b\\ 



p p 



E 

(=1 
p-i p 



TT n^^\ ! I IM n Lnixi - X2) ■■■ Ln{x2p - Xl)dx 

^^J[-^,^]2A^^{\x2^\^\x2^-l\Y-'' 



E 



\hj{x2j-l)\\x2j-l - X2j\Ln{xi - X2) . . .i„(x2p - 



< C 



fM\\b\\ 



V TO 



3pa 



p-1 



''+^dx\ +CL(^^ffl^logn) (logn)2f-i^|i5,|i2. 



□ 



Lemma E.6. Let f = l/(47r^/), and let p > 1/2 a^d L > 1, t/ien under the 
conditions of Lemma \E.4\ we /iat;e the following alternative bound for (|E.4p ; 



i=2 



where 



TT ^2 

/2 



2 {x)dx I + error 



(E.7) 



error < L3/4n(i-3p)/4 J| ^(20 + ^ /l(2^);^^^^7m(^ J] Af '^O 



1=1 



Remark E.l. The constant appearing on the right hand side of (jE.7P depends 
on M and m, but in all our applications of Lemma \E.6l the constants M and 
m will be bounded and of no consequence. 
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Proof. Following the construction of iDahlhausI (|l989t ). equation (13), we write 
|tr{n^=i T-i(/)T„(/2,)} - tr{n^^i r„(/)r„(/2,)}| as 



















tr|(Ai 






1=2 



En 

j=2 



1=3 + 1 



(E.l 



where A, = T„^(/2,_2)T-i(/)T|(/2j), B, = {f2j-2)TM)T^ {f2j) and /o := 
f2p (similarly for po, L^°^ and M(°)). When j = p, the factor flLj+i is 
understood to be the identity. Without loss of generality, the functions f2j are 
assumed to be positive (it suffices to write f2j ~ f2j+ ~ f2j-)- Lemma IE.3I 
implies that for each j, 



\TnHf)T^{.f2m 



< 



,((i23-d) + +e 



(E.9) 



Using the relations in (1.6) (main paper) it then follows that 

II n ^'1^ n \\Thf2i)TnHf)f 

p-1 



(E.IO) 



First we treat the term {Ai - Bi) Y^i^2 ^^e right in (jE.Sp . Writing R 
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/„ - tJ (/)r„(/)ri (/), it follows that 



tr 



tr 

1=2 

TrI if2p)TnHf)RT-Hf)TM2)TnHf)T-Hf)T^ (U) [] A 

1=3 

<\R\\T-^f)Tn{f2)T-Hf)\\ I ri {f2p)T- ^ (/) 1 1 1 1 T~ ^ (/)Ti (/4) 1 1 1 1 n ^. 
<i^„(i-p)/2+.+H29 J|m(20 f r ^Mrfa, + error) 



1=3 



1=2 

§^dx + ^3/2 / ^,^(2)\ ' „(l-3rt/2 ^ m(2)^(2)^-p. 



(E.ll) 



The first inequality follows from the relations in (1.6) (main paper). The second 

_l _2_ 

inequality follows after writing |T„ ^ (/)T'„(/2)T„ ^ (/)| as the sum of a limiting 
integral and an approximation error; in addition we use (jE.9[) and Lemma |E.2| 
by which 

|i?|^ < KL{M/m)^n^-P+' < Ln^-P+\ (E.12) 

This follows from Lemma lE. 41 which we use to bound the approximation error. 
The second term within the brackets in (jE.lip constitutes part of the term 
error. 

Next we bound the term (HCi Bi){Aj - B^) ULj+i for j = 2. 

Similar to the preceding decomposition, we have 



tr 



Bi{A2 - B2)\[Ai 



1=3 



tr 



1=3 . 
p 

< I SiTi (/2 )r~ ^ (/) 1 1 i?| 1 1 T- Hf)Thf4)\\\\l[Ai\\. 

1=3 

The terms \R\, \\TnHf)Thf4)\\ and || nr=3-4z|| are bounded as in (|R9|) . (|e10|) 
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and (jE.12|) . For the term \BiTn {f2)Tn ^ (/)| we have the decomposition 
|Siri(/2)T,7^(/)|' = tr \T---{f)TS{h)B\BiTk{h)Tn~^{f) 



tr 



= tr 



B\B^T^{h)T,,{f)T?,{f2) 



tr 



BlBiT^{h)Tn m)RTn Hf)T^if2) 



< \B,T^ {f2)T^ {fY + \BlB,\\R\\\T^ {f2)Tn ^f)f. 

Using again Lemmas IE. 11 IE.2I and IE.31 we find that the first term on the right 
is bounded by 

and the second term by 



7T f2 



Consequently, 



tr 



Bi{A2-B2)'[[Ai 



1=3 



p— 1 

1 = 2 Li-TT / {X) 



\J-n r{x) 



Note tliat 



dx\ <M(2)Af(2p), 



/|(a;)/2p(a;) 

(Li(2)^,^(2p))l/2(^,^(2))3/2^-(p+p.)/2 < i(M(2))2„-P + L(2)m(2)m(2p)„-''^ 

and L?!^'' < i'^/2,^(i-3p)/2^ Therefore tlie terms on tlie right are of the same 
order as the right hand side of (|E.lip . A similar argument applies to the term 

(LL(2p)jl^(2p)-)l/2^^,j(2)-)2^-(p+P2p)/2^ 

Finally, we bound the term (HCi Bi){Aj-Bj) ULj+i in for 3 > 3- 
For j > 3, Lemma FE. II implies that 



1=1 



P{x) 



dx + error j 
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where 

error, < n^+2EL-^(rf2.-d)+ J| ^^(21)^^(21-2) L-p ^ ^ A!^„-P2.)' 

1=1 \ 1=1 / 

Consequently, we have for all j > 2 



tr 



J— 1 p 

<in^'ll^l n P'lll|T^"(/2,)7^n^(/)||||Ti(/(2,_2))T,7^(/)|| 



< i5n5+(l-P)/2+2<?+e 

for all j > 3, which finishes the proof of Lemma FE. 61 □ 

F Holder constants of various functions 

Lemma F.l. Let 60 G Q{(!i,Lo). Then fo satisfies condition (|E.2[) with p = 1 
when /3 > |, and with any p < /3 ~ ^ when /3 < |. The Holder- constant 
only depends on Lq- When 6 £ Qk{l3,L), fd,k,e satisfies (|E.2p with p ~ 1, 
regardless of /3. The Holder- constant is of order The function — log(2 — 

2 cos{x)) fd.k.e satisfies condition (jE.ip with p = 1 and Holder- constant of order 

ki~^. The functions Gk fd.k.e o,'>^d H^ fd.k.e, withGk and Hk as in (jP.ip . satisfy 
(jE.ip with p = 1 and Holder- constant of order k. 



Proof. The function X]j!Lo cos(ja;) (i.e. the logarithm of the short-memory 
part of /o), has smoothness p < /3 — i, since 



00 



E I^ojII cosOx) - cosOy)| < E \x - y\' 

J=0 \j=0 



which is finite only when p < — 5- Since X)jlo I^jI ^ when 9 € 6(/? — 
1/2, L) and /3 > 1, the functions X^jlo cos(ja;) and exp{^°^p 6'j cos(jx)} have 
the same smoothness; only the values of L and M differ. The same calculation 
can be made when the FEXP-expansion is finite: when 9 E then for 



all x,y G [— tTjTt], 



E'^J 



{cos{jx) - cos{jy)) 



k 

<\x-y\Y,j\9,\<^ki-^\x-y\. (F.l) 
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Since 



k 

|Gfe(x)-Gfc(2/)| < 2^77,1 cos(ja;)-cos(jy)| =0(fc)|a;-y|, (F.2) 

Gkfd,k,e has Holder-smoothness p = 1, its Holder-constant being 0{k). The 
same result holds for Hkfd,k,e, since Hk{x) = — log(2 — 2cos(a;)) — Gk{x) (see 
dElJ) and fci-'^ = o{k) for all P > I. □ 



G Proof of Lemma B.2 



For easy reference we first restate the result. Let W„{d) denote any of the 
quadratic forms 

X'T-\fdM)Ba{d, ed,k)X - tr %{fo)T-\fd,k)B,{d, OdM)] 

in (B.2) (in the main paper). Then for any j < J, {li, ■ ■ ■ ,lj) G {0, . . . , k}^ and 
a € S{li, . . . we have 

\Waid)-W„ido)\^ op J\d-do\n^+'k-^), (G.l) 

tr [B,{d,9d^k)]~tT [B,id,,§dJ] 

= (d-do)tr[Ti,,(do,fc)] + (d-d„)2o(72^+U-5+(i-^*/2)+) (G.2) 
= (d - do)tr[Ti^,(do, k)] + {d- dofo{n^-^/k), 

tr [{TMo)T-\!d.k) - In)BAd,0d,k)] - tr [{TMo)T;;\fd^,k) - In)B„{do,9d^,k)] 
= {d- doMT2 a{do, k)] + id~ do)^o{n/k) + {d - do)o{n'+h-i). 

(G.3) 

Proof of Lemma B.2. We first prove (jG.2p . Developing the left-hand side in d 
we obtain, for all j, {li, . . . ,lj) E {0, . . . , k}^ and a E Sj, 



tr [B„{d,edM)] - tr [B,{do,9dJ] 
= {d-do)tr [B'^{do,9d.,k)] + 



[d-dof 



tr 



(G.4) 



where d E (0?, do), and B' and i? denote the first and second derivative with 
respect to d, respectively. Writing 

B„(.^{d,k) = T^{HkV„(rifd.k)T-Hfd,k) 

— Tn{'^a{i)fd,k)Tn^{fd,k)Tn{Hkfd,k)Tn ^{fd,k), 

it follows that B^{d,9d.k) equals 
kl 

Bl{dJd,k)^J2U'^"(^-infd,k)T-\fd,k)B,i^)id,k)]lT^^^ 

i=l j<i j>i 
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We recall the definition of Ti ^ in Lemma B.l (main paper), and conclude that 
B^{d,6d.k) = Ti^„{d,k). Consequently, the first term on the right in (|G.4|) 
equals {d ~ do)tr[Ti^a-ido, k)]. 

The second derivative B^{d,0d,k) equals 

kl 

<n ii 



X 



1\<]<11 



kl 

1=1 j<i i<j 



S^(d,6'd.fc)j = o(n<^+5fc-5+(i-/3/2)+). FromLemmaEI 



We now show that tr 

and the above expression for B^(d,6d^k), it can be seen that tr B^{d,9d^k) 
converges to zero. To bound the approximation error, we cannot use directly 
Lemma [£.41 because the bound in (|E.4|) becomes too large when (3 < 2 and \(t\ 
is larger than 1. We therefore use Lemmas IE. II and IE. 61 Let A^{d,9d,k) be the 
matrix obtained after replacing every factor T;;''-{fd,k) in B^ {d, Od^k) by Tnifd,k), 
for fd.k ~ fdk/i'^'^'^)- recall from Lemma [F.ll that the Lipschitz constant 
of fd^k is 0(fc(2-/3)+), and for Hlfd,k and HlV ,(^,n)fd,k (m < k|, j = 1,2) it is 
O(fclogfc). Consequently, Lemma FE. II implies that 



tr 



when k < A:„ and {3 > 1. It follows from Lemma [£.61 that 



Kid,9d,k)\ = 0(fcn^) = o(n^+U-5+(i-/5/2)+) 



tr 



Kid,9d,k)\ -tr [B';,{d,9d,k)\ =0(ni/2+^fc(i-^/2)^) 

= o(n'+U--H(i-/3/2)+) 



Hl{x)dx 



Note that in the case where B^{d,9d,k) contains a Toeplitz matrix of the form 
Tn{H^fd.k) or T'„(7J|Vo.(m)/d,fc) then it contains no other Toeplitz matrix in- 
volving Hk and we can set /2 ~ Hkfd.k or /2 = H'^V cr{ni)fd.k and use Remark 
2.1; this leads to the above error rate. Combining the preceding results for 
\tT[A'^{d,9d,k)]\ and \tr[A'^{d,9d,k)] - tr[B'^id,9dM)]\ we obtain that 



tr 



BlidJd^k)] = o(n'+3fc-^+(i-^/2)+)^^(^i-5/^)^ 



which completes the proof of (|G.2p . 

Next, we prove (|G.3p . Writing - fd.k = fo- fd^.k + fd„,k - fd.k, it follows 
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that the left-hand side of (|G.3[) equals 

tr [T„ {fo - fd,k)T-'{fd,k)B^{d,ed^k)] - tr [r„(/o - /d„,fc)r-i(/d„,fc)Ba(do, 
= tr [TMo - fd^.k) {T-\fd,k)B,{d, 0d,k) - T-Hfd^.k)B„{do, 
+ tr [T„(/d„.fc - fd^k)T~\fd,k)B^{d, Od^k)] 
:=Ci+C2. 

Using p.4p we write fd,k = fdj,,ke^'^~^ and /„ = /d„,fce^<'° ^ and we develop 
C<T(d, 0d,fc) = T-'^{fd,k)Ba{d, 9d,k) around d = do- It follows that 

Ci = tr [T„{fo ~ fd^M) {T-\fd,k)B^{d,0d,k) - T-'{fd^,k)B^{do,0d^M)}] 

= {d- d,)tT \tMo - fd^M)Cl{do, 0d^,k) 



+ {d~dof (l-u)tr TMo~ fd^,k)C,{du,ed^^k) 



du. 



with du ~ ud + {1 — u)do. For the first term on the right, we write, using 
Lemmas lE.ll and IE.61 



tr 



Tnifo — fdo,k)C^{do, Od„,k) 

n r fo- dd,,k 



Hk{x) cos{lix) . . . cos{l\a-\x)dx + I 



27'' J-Tl fda,k 

where cr is a partition of {1, and the error term is 



O [\\Ad^,k\\oonHk + fc0-5(3/2-«+ + fcO.5(3/2-0), ^ fc|l A,„,,|l, 



oo , 



which is o(/c -^/^n^/^ Similarly. Lemmas lE.ll and lE.6l implv that there exists 
c £ M such that for al d S {do — Vn, do + Vn) 



tr 



27r J-TT fda,k 
where the error term is of order 



Tnifo — fda,k)^a (^d.k) 

' ° °' \ff^(a;) cos(Zix) ... cos(Z|cr|a;)da; + error. 



O [\\Ad^A\oon^{k + + k-^'^'-'^'H^ + fell A,„,fe||oo)^) - o (^^) . 

This implies that Ci = 0{Sn{d)). 

Using a Taylor expansion of Ca{d,9d,k) and of g,-id—da)Hi, ^j-q^j^jJ fi^_ 
follows that 

C2 = -(d-do)tr [TMd.MHk)T-\fd^^k)B,{do,0d.,k)] 

- ^{d-do)hi \T^{fd^,kHle-'^'^ -''"^"'^)CM' .dd'.k) + 2TMd^MHk)C'Ad',0d',k) 
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for some d between d and do- The first term equals tr[r2,cr]- The second equals 



-{d - doftr [TMci'^HDC^id', ed-,k) + 2T„{fd^^kHk)CUd\0d'.k) 



n(d — doY 
"hi 



H^{x) cos(/ia;) . . . cos{l^„^x)dx + error, 



where the error term is O (n" (k + (nfc"^ + fcn^)^/^)) = o(fc"^n^"*). 

Therefore 

C2 = {d - do)tr[T2A + 0{n/k). 

Finally, to prove jSU, let Z = T~^(/„)X and let Aa = tI {f,)T-\fd,k)BM, 0d,k)Thfo). 
Then for any \d — do\ < Vn, we have 

W^{d) - WM = Z*iAd - AdJZ - ti-{Ad -Adj. 

Writing A'^ for the derivative of Ad with respect to d, it follows that 

Ad-Ad^ = {d-do)A'^, (G.5) 

for some d between d and dg. Using (|D.6|) . we find that 

= ri(/o)T-i(/d,fe)r4iffe/d,fe)r-i(/d,,OT4B.(d,0"d,,O)rrh/o) 
+ ri {fo)T-\fdM)B'M, Od.k)TS (/o). 

Therefore, Lemma 2 of iLieberman et all ( 2011 ) and the inequalities in (1.6) 
(main paper) imply that 



\Ad-AdA < \d-do\\A^\ 



kl 



< C|d-d,|||T|(/„)T„ M/d>)ll'nil^" ^fd,k)B.i^)id,Od,k)T^{l 



d,k) 



+ \Tn md,k)Tn{HuV.(,)fd,k)Tn Hfd,k)\ 

= \d-dM^O {\T-Hfd,jT^{Hkfd,k)TnHfd,k)\ + \TnHfd,k)Tn{Hkyai.)fdM)TnHfd,k)\ 

(G.6) 

where (7(1) can also be the empty set, in which case a(i)fdk = fdk- We bound 
the terms between brackets using Lemma FE. 41 with p = 2, f ~ fdk ^^^'^ 5i — 52 
equalling either Hkfdk or Hk'^ a(i)fd k- The Holder constants of these functions 
are given by Lemma IF. II Hence we find that 



(G.7) 



\Tn HfdM)Tn{Hk.fdM)Tn = tr (T"! (/,-, )T„ (iJfe /,;,)) ' 



^ / i/2(x)dx + 0(n^(fc + fc2-^)) = 0(ni-i/(2«(logn)i/(2«). 
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The last inequality follows from equation (|D.7p in Lemma fP. II and the fact that 
k = kn and /3 > 1. Similarly, it follows that 

|7^."^(/.-.fe)7^n(i/fcV.(,;)/,-,)T-^(/,-,fc)|' = 0(ni-i/(2«(logn)i/(2«). (G.8) 

Inserting (|GT6)) . (|GJ| and ((08)) in ((G3)) . we find that \ Ad - Ad J < \d - 
do\'n}^^~^^^'^^^'^'^ , for all \d — do\ < Vn and all e > 0, when n is large enough. 
Consequently, we can apply Lemma 1.3 with A = {Ad — Ad^)/\Ad — Ad^\, so 
that when n is large enough 

sup (\W,{d) - W,{do)\ >\d- do\n^'+^-^) < e-"'/^ (G.9) 

|d-rfo|<C„ ^ ^ 

Using the above computations with jd — d'| < n^^, we obtain 

\W^{d) ~ W^{d')\ < ^ z*Z) . 

Hence, for all e < i and c > 0, 

Po I sup \W„{d) - W„{d')\ >n-A < Po {Z'Z > n^-^') < e"™, (G.IO) 

provided n is large enough. Hence, we obtain (|G.1[) by combining (|G.9[) and 
l|G.10|) in a simple chaining argument over the interval {do — u„, do + 

□ 
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